> There is a simpler and more honest alternative: set a clear academic threshold for readiness and then admit students by lottery.
What minimum SAT score are you imagining here? There aren't that many students in the country who score 1550+ (though you could argue that would change as admissions criteria changed of course). I'm confident that fewer than 100k students would score a 1550+ on the SAT and a 5 on three AP exams, for example—and this is assuming they know those to be the only admissions criteria for eligibility in the lottery. I made some estimates here: https://arjunpanickssery.substack.com/p/college-admissions-doesnt-need-to
In general, I don't understand the fascination with lottery systems, which prioritize a confused notion of fairness—unless the goal is merely to reduce the amount of effort and other resources students spend on the admissions contest, in which case you should make that explicit as the goal and try to quantify the costs and benefits that you're trading off.
> Once that bar is met, selection by lottery acknowledges an uncomfortable, but honest truth: beyond readiness, differences among applicants are often too small, too subjective, or too context-dependent to rank meaningfully ... Faced with thousands of academically capable applicants, institutions insist on ranking the unrankable
Your implied argument is that it's literally impossible to distinguish the top e.g. 10k students (out of a graduating class of 4 million) from the top 100k students. But you don't defend this assumption at all.
Reducing the amount of effort and other resources students spend on the admissions contest is a worthy goal in and of itself. Give our bright high schoolers their lives back.
Are you going to allow departments to do their own admissions? Outside of the most selective institutions, some departments require far more capable students than the general student admission bar - and institutions want those most capable students. I did my degrees in Physics and Engineering. I did my BS at an open admissions state university, but the Physics for Physicists class started with ~ 200 students the freshman year - and was down to about 5 by the end of the sophomore year. The math department similarly filtered the math students who chose to major in math. Such filtering is probably still universal in the STEM areas.
I suspect demonstrated interest should matter more. It’s harder for some students than others to visit or credibly express interest (a problem that could be worked on), but it’s a win-win for both students and colleges to have students who actually want to go there.
I agree with most of your particular points—most of the holistic admissions stuff is silly—but I’m not sure about the frame, which seems to be: colleges should be meritocracies where measures are objective, and all remaining noise, of which there is a great deal, will be handled purely by luck. I don’t think I’d want to operate, or apply to, a college run in that way. I’m not sure that this wouldn’t be doubling down on what is ossified and broken about college.
In the new lotteried equilibrium, there's a good chance that matters of interest would at least largely take care of themselves. Instead of needing to apply to very many colleges to hedge bets, where you were secretly much more interested in some than others but attempting to signal interest to all, you could know just statistically how many you needed to apply to, with your own stats, to have a great shot of getting into somewhere satisfactory. And presumably it would be much easier and quicker to run multiple rounds of admissions, not just early decision vs. regular. Willingness to commit earlier rather than later of course filters for interest. Perhaps colleges could tick up the objective bar as the admissions year wears on, even.
In my reading, part of the problem addressed by the post is that students are currently forced to spend their formative years jumping through many costly hoops in order to "signal their interest". The credible signals are the essays and extracurriculars and all the rest of the insane admission process--who would go through all that if they weren't interested?
Lotteries are a wildly underrated policy tool and not just public policy. It would also cut through the problems with things like Taylor Swift or Super Bowl tickets or whatever.
I think part of the problem is that this arises in areas of super high demand, which is to say things people really really badly want, and there's just some weird American psychology thing with leaving that up to chance and not letting people go to insane lengths to "buy" it.
But it's such a big and wide-ranging problem and lotteries are such a killer solution that even dipping the toe in (give enrollment to PART of the class via lottery, whatever) would bring a lot of benefit.
The problem with this proposal is that it proceeds from a radically different understanding of admissions merit than elite colleges are operating under. These institutions see their role as hand selecting a future elite, not rewarding some subset of the brightest high school students.
Joshua, for those of us who believe in this approach, do you know of any promising leads for something like this actually happening — if not at scale, then at least at some enterprising college or university, to demonstrate its advantages? (Or, is someone already doing this?)
A side note could you define what "HALO students" means? (I'm embarrassed to say that, despite a decade of working in test-prep, I hadn't heard this! And a light Googling didn't come up with any answers...)
HALO means High-Achieving Low-Income (now edited into the post!), and colleges — for a lot of reasons — are all likely intuitively opposed to ever implementing such a policy. But it would definitely be to these institutions' benefit, which is the ironic part! We are looking into this though
Why do you say only wealthy people can "take advantage" of test prep and retaking the SAT? College Board has a lot of free prep material, and it only costs $68 to take the SAT.
I certainly agree that holistic admissions are a serious problem, but lotteries are not the answer either. The results would be especially weird at selective but not highly elite schools where the difference between the very best students and those right at a minimum threshold may be enormous. Does it really make sense for a flagship state college to admit a student with a 1200 SAT and 3.5 GPA over a student with a 1550 SAT and 3.9 GPA because the first student won their lottery? 350 SAT points is not noise; it’s a large and objective signal. Colleges should use genuine academic signals for admissions, and they should do so in transparent, repeatable ways. Putting meaningful weight on an ever growing hodge hodge of weak academic signals (and frankly, non-signals) within a black box algorithm is what needs to be ended. I will also note that universal lottery adoption will encourage mass applying by half decent students all over the place to increase the odds of getting in somewhere. The effect would be to further drop the odds of winning at any one school, triggering even more applications in a vicious cycle.
I recommend looking closely at university admission systems around the world for some inspiration here. Perhaps this would be a good topic for a future article. In Canada, back in the nineties (when I was applying), the system for almost all university admissions was a pure grade ranking. There are some important caveats to that. Students applied and were admitted to a particular faculty within a university (i.e., arts, science, engineering) and those faculties used different courses in their GPA calculation. For example, the engineering faculty was going to use your English, math, physics and chemistry marks for grades 11 and 12. At the time I was in high school, many provinces had provincial standardized final exams associated with each subject. In BC, these were weighted 40% of your final grade. The existence of these exams greatly dampened grade inflation from the get-go, as large school or instructor level differences between school assigned grades and exam grades in a given subject were visible and scrutinized. Curriculum was also quite standardized within each province. Ultimately each faculty just ran a calculation of grade averages for the relevant subjects, ranked the students from top to bottom, then admitted everyone above a cutoff based on number of available places and historical acceptance rates. Prior year cutoffs were published, so you had a sense of your chances.
A few very competitive programs required reference letters from teachers, a few paragraphs about why you were interested in this program and/or a list of relevant accomplishments outside class. This was unusual for admission (as opposed to scholarships). On the subject of accomplishments, it was also pretty clear that these programs were typically less interested in your volunteer work or club participation compared to your scores on math contests or some similar academic award.
A final interesting point on Canada: as provincial exams disappeared over the last twenty years, grade inflation has become rampant and admissions have become more holistic, looking more and more like the byzantine US system (references, essays, extracurriculars). Here is one private school principal’s rant on the subject: https://www.visst.ca/blog/university-admissions-and-provincial-exams. This reinforces that retaining at least one standardized test behind admissions is essential. And if you have that, then you really don’t need much more for all but the most elite schools.
So what reforms to competitive college admission could help in the US? Here are some additional ideas:
- Ditch the personal essays. It was always a bit dubious how much help students had received and how honest their content. Now with LLMs, this admission metric has reached the pinnacle of futility. The only essay worth the paper it’s written on is a proctored essay. Schools who feel they need a writing sample should insist that students appear at a qualified testing center to write one (they might also use their alumni to proctor essay writing rather than conduct inconsistent interviews).
- End games around early admissions. Schools who receive any federal funding should be required to adhere to some uniform rules on application, decision and acceptance dates.
- Publish transparent formulas for admissions decisions that are a weighted average of one or more standardized tests and student grades. This is enough for the vast majority of schools to work with. Very competitive schools might add a third explicit modifier for achievements/activities outside class with a limit of perhaps 2 or 3 items and a clear rubric for how they are evaluating and verifying these. The incentive for students should be focused participation on a few extracurriculars. Paid employment should be valued on an equal footing with sports, volunteering and other productive, non-academic endeavors.
- Fight grade inflation by demanding class ranks, putting in place more subject level standardized tests (especially offering a level below AP classes), and explicitly adjusting student grades based on a schools’ average performance on objective measures like the SAT/ACT, state exams or even historical performance of their students at the university (I believe Waterloo in Canada used to adjust student grades based on how well students from that school performed in 1st year Calculus).
- Defend and expand standardized testing. Not only is this the least biased measure of how students will perform once in college, its remaining biases are the easiest to combat. The money involved in testing is peanuts compared to what sports and other activities can suck up. And having freed students from hundreds of hours previously spent writing personal essays and resume-padding extracurriculars, they now have plenty of time to do some test prep and to write additional standardized tests or contests. Press for changes to testing that will even the playing field for those with fewer resources:
- Expand free PSAT testing in 10th grade in all states and push for universal administration. Find the students early who have any plausible chance of being competitive college candidates and mail them what they need to know about how to access free/cheap test prep, testing, recommendations to take the SAT twice, etc.
- Schools should adopt more sensible use of multiple test results, for example, always averaging the two most recent tests (or taking the best of the most recent two), that put less advantage on excessive re-testing.
- Push test designers to roll back the test changes over the decades that have made exams like the SAT somewhat more easily prepped and compressed at the top of the range. These changes were sold as closing racial and socio-economic gaps. They haven’t even accomplished that, so on what basis can they be defended? With the exam moving all digital, it is easier than ever to administer an adaptive test where a perfect score is never obtained. Digital also allows you to move away from pure multiple choice format in ways that are likely to also make the exam less easily prepped.
- Fight to end extra time accommodations — in court if necessary. This is becoming madness. Ideally the ADA would be amended to strictly limit accommodations for evaluations in education. Otherwise test companies should see it as a threat to their credibility and start designing to defeat it. They might design adaptive digital tests that just plain get harder, give everyone plenty of time then hold the line on any time accommodations.
Interestingly, your suggestion is more like what China does for college admissions-- your college and even your major are almost entirely based on your scores on the gaokao. It's got its own challenges, but it's much more equitable than our system of legacies and fuzzy metrics.
I love your idea of a blind experiment to see the extent to which admissions officers choose the same students. In "Who gets in and Why" by Selingo, he makes the case that admissions to these prestigious universities are already basically a lottery.
> There is a simpler and more honest alternative: set a clear academic threshold for readiness and then admit students by lottery.
What minimum SAT score are you imagining here? There aren't that many students in the country who score 1550+ (though you could argue that would change as admissions criteria changed of course). I'm confident that fewer than 100k students would score a 1550+ on the SAT and a 5 on three AP exams, for example—and this is assuming they know those to be the only admissions criteria for eligibility in the lottery. I made some estimates here: https://arjunpanickssery.substack.com/p/college-admissions-doesnt-need-to
In general, I don't understand the fascination with lottery systems, which prioritize a confused notion of fairness—unless the goal is merely to reduce the amount of effort and other resources students spend on the admissions contest, in which case you should make that explicit as the goal and try to quantify the costs and benefits that you're trading off.
> Once that bar is met, selection by lottery acknowledges an uncomfortable, but honest truth: beyond readiness, differences among applicants are often too small, too subjective, or too context-dependent to rank meaningfully ... Faced with thousands of academically capable applicants, institutions insist on ranking the unrankable
Your implied argument is that it's literally impossible to distinguish the top e.g. 10k students (out of a graduating class of 4 million) from the top 100k students. But you don't defend this assumption at all.
Reducing the amount of effort and other resources students spend on the admissions contest is a worthy goal in and of itself. Give our bright high schoolers their lives back.
Are you going to allow departments to do their own admissions? Outside of the most selective institutions, some departments require far more capable students than the general student admission bar - and institutions want those most capable students. I did my degrees in Physics and Engineering. I did my BS at an open admissions state university, but the Physics for Physicists class started with ~ 200 students the freshman year - and was down to about 5 by the end of the sophomore year. The math department similarly filtered the math students who chose to major in math. Such filtering is probably still universal in the STEM areas.
I suspect demonstrated interest should matter more. It’s harder for some students than others to visit or credibly express interest (a problem that could be worked on), but it’s a win-win for both students and colleges to have students who actually want to go there.
I agree with most of your particular points—most of the holistic admissions stuff is silly—but I’m not sure about the frame, which seems to be: colleges should be meritocracies where measures are objective, and all remaining noise, of which there is a great deal, will be handled purely by luck. I don’t think I’d want to operate, or apply to, a college run in that way. I’m not sure that this wouldn’t be doubling down on what is ossified and broken about college.
I would co-sign this one and have said as much (very briefly) before myself: https://x.com/gtmom/status/1959408291022295179
In the new lotteried equilibrium, there's a good chance that matters of interest would at least largely take care of themselves. Instead of needing to apply to very many colleges to hedge bets, where you were secretly much more interested in some than others but attempting to signal interest to all, you could know just statistically how many you needed to apply to, with your own stats, to have a great shot of getting into somewhere satisfactory. And presumably it would be much easier and quicker to run multiple rounds of admissions, not just early decision vs. regular. Willingness to commit earlier rather than later of course filters for interest. Perhaps colleges could tick up the objective bar as the admissions year wears on, even.
A matching system could handle this?
Came here to say the exact same thing. A matching system feels like a much better fit for this problem than a lottery.
In my reading, part of the problem addressed by the post is that students are currently forced to spend their formative years jumping through many costly hoops in order to "signal their interest". The credible signals are the essays and extracurriculars and all the rest of the insane admission process--who would go through all that if they weren't interested?
Lotteries are a wildly underrated policy tool and not just public policy. It would also cut through the problems with things like Taylor Swift or Super Bowl tickets or whatever.
I think part of the problem is that this arises in areas of super high demand, which is to say things people really really badly want, and there's just some weird American psychology thing with leaving that up to chance and not letting people go to insane lengths to "buy" it.
But it's such a big and wide-ranging problem and lotteries are such a killer solution that even dipping the toe in (give enrollment to PART of the class via lottery, whatever) would bring a lot of benefit.
The problem with this proposal is that it proceeds from a radically different understanding of admissions merit than elite colleges are operating under. These institutions see their role as hand selecting a future elite, not rewarding some subset of the brightest high school students.
Joshua, for those of us who believe in this approach, do you know of any promising leads for something like this actually happening — if not at scale, then at least at some enterprising college or university, to demonstrate its advantages? (Or, is someone already doing this?)
A side note could you define what "HALO students" means? (I'm embarrassed to say that, despite a decade of working in test-prep, I hadn't heard this! And a light Googling didn't come up with any answers...)
HALO means High-Achieving Low-Income (now edited into the post!), and colleges — for a lot of reasons — are all likely intuitively opposed to ever implementing such a policy. But it would definitely be to these institutions' benefit, which is the ironic part! We are looking into this though
Why do you say only wealthy people can "take advantage" of test prep and retaking the SAT? College Board has a lot of free prep material, and it only costs $68 to take the SAT.
I certainly agree that holistic admissions are a serious problem, but lotteries are not the answer either. The results would be especially weird at selective but not highly elite schools where the difference between the very best students and those right at a minimum threshold may be enormous. Does it really make sense for a flagship state college to admit a student with a 1200 SAT and 3.5 GPA over a student with a 1550 SAT and 3.9 GPA because the first student won their lottery? 350 SAT points is not noise; it’s a large and objective signal. Colleges should use genuine academic signals for admissions, and they should do so in transparent, repeatable ways. Putting meaningful weight on an ever growing hodge hodge of weak academic signals (and frankly, non-signals) within a black box algorithm is what needs to be ended. I will also note that universal lottery adoption will encourage mass applying by half decent students all over the place to increase the odds of getting in somewhere. The effect would be to further drop the odds of winning at any one school, triggering even more applications in a vicious cycle.
I recommend looking closely at university admission systems around the world for some inspiration here. Perhaps this would be a good topic for a future article. In Canada, back in the nineties (when I was applying), the system for almost all university admissions was a pure grade ranking. There are some important caveats to that. Students applied and were admitted to a particular faculty within a university (i.e., arts, science, engineering) and those faculties used different courses in their GPA calculation. For example, the engineering faculty was going to use your English, math, physics and chemistry marks for grades 11 and 12. At the time I was in high school, many provinces had provincial standardized final exams associated with each subject. In BC, these were weighted 40% of your final grade. The existence of these exams greatly dampened grade inflation from the get-go, as large school or instructor level differences between school assigned grades and exam grades in a given subject were visible and scrutinized. Curriculum was also quite standardized within each province. Ultimately each faculty just ran a calculation of grade averages for the relevant subjects, ranked the students from top to bottom, then admitted everyone above a cutoff based on number of available places and historical acceptance rates. Prior year cutoffs were published, so you had a sense of your chances.
A few very competitive programs required reference letters from teachers, a few paragraphs about why you were interested in this program and/or a list of relevant accomplishments outside class. This was unusual for admission (as opposed to scholarships). On the subject of accomplishments, it was also pretty clear that these programs were typically less interested in your volunteer work or club participation compared to your scores on math contests or some similar academic award.
A final interesting point on Canada: as provincial exams disappeared over the last twenty years, grade inflation has become rampant and admissions have become more holistic, looking more and more like the byzantine US system (references, essays, extracurriculars). Here is one private school principal’s rant on the subject: https://www.visst.ca/blog/university-admissions-and-provincial-exams. This reinforces that retaining at least one standardized test behind admissions is essential. And if you have that, then you really don’t need much more for all but the most elite schools.
So what reforms to competitive college admission could help in the US? Here are some additional ideas:
- Ditch the personal essays. It was always a bit dubious how much help students had received and how honest their content. Now with LLMs, this admission metric has reached the pinnacle of futility. The only essay worth the paper it’s written on is a proctored essay. Schools who feel they need a writing sample should insist that students appear at a qualified testing center to write one (they might also use their alumni to proctor essay writing rather than conduct inconsistent interviews).
- End games around early admissions. Schools who receive any federal funding should be required to adhere to some uniform rules on application, decision and acceptance dates.
- Publish transparent formulas for admissions decisions that are a weighted average of one or more standardized tests and student grades. This is enough for the vast majority of schools to work with. Very competitive schools might add a third explicit modifier for achievements/activities outside class with a limit of perhaps 2 or 3 items and a clear rubric for how they are evaluating and verifying these. The incentive for students should be focused participation on a few extracurriculars. Paid employment should be valued on an equal footing with sports, volunteering and other productive, non-academic endeavors.
- Fight grade inflation by demanding class ranks, putting in place more subject level standardized tests (especially offering a level below AP classes), and explicitly adjusting student grades based on a schools’ average performance on objective measures like the SAT/ACT, state exams or even historical performance of their students at the university (I believe Waterloo in Canada used to adjust student grades based on how well students from that school performed in 1st year Calculus).
- Defend and expand standardized testing. Not only is this the least biased measure of how students will perform once in college, its remaining biases are the easiest to combat. The money involved in testing is peanuts compared to what sports and other activities can suck up. And having freed students from hundreds of hours previously spent writing personal essays and resume-padding extracurriculars, they now have plenty of time to do some test prep and to write additional standardized tests or contests. Press for changes to testing that will even the playing field for those with fewer resources:
- Expand free PSAT testing in 10th grade in all states and push for universal administration. Find the students early who have any plausible chance of being competitive college candidates and mail them what they need to know about how to access free/cheap test prep, testing, recommendations to take the SAT twice, etc.
- Schools should adopt more sensible use of multiple test results, for example, always averaging the two most recent tests (or taking the best of the most recent two), that put less advantage on excessive re-testing.
- Push test designers to roll back the test changes over the decades that have made exams like the SAT somewhat more easily prepped and compressed at the top of the range. These changes were sold as closing racial and socio-economic gaps. They haven’t even accomplished that, so on what basis can they be defended? With the exam moving all digital, it is easier than ever to administer an adaptive test where a perfect score is never obtained. Digital also allows you to move away from pure multiple choice format in ways that are likely to also make the exam less easily prepped.
- Fight to end extra time accommodations — in court if necessary. This is becoming madness. Ideally the ADA would be amended to strictly limit accommodations for evaluations in education. Otherwise test companies should see it as a threat to their credibility and start designing to defeat it. They might design adaptive digital tests that just plain get harder, give everyone plenty of time then hold the line on any time accommodations.
Interestingly, your suggestion is more like what China does for college admissions-- your college and even your major are almost entirely based on your scores on the gaokao. It's got its own challenges, but it's much more equitable than our system of legacies and fuzzy metrics.
https://scceichinabriefs.substack.com/p/the-highest-exam-how-the-gaokao-shapes
I love your idea of a blind experiment to see the extent to which admissions officers choose the same students. In "Who gets in and Why" by Selingo, he makes the case that admissions to these prestigious universities are already basically a lottery.