There was a terrifying article in the New York Times the other day. It appears that Professor Noriko Arai is trying to program a computer to pass the Tokyo University admissions exam.
If she succeeds, it would be a very big deal. Right now, the computer is stumped by natural language questions and historical context. But those might be automatable. If they are, then the implication is that huge swathes of human activity will lose their market value.
The corollary would seem to be that if a computer can pass the Tokyo University entrance exam, then that exam is no longer a useful measure of human merit.
Admission to New York City’s specialized high school is based on the Specialized High Schools Admissions Test. You can find a description and sample exam at this link. To this lay reader, the scrambled paragraphs and reading sections seem like they would be very hard to program, the math seems trivially easy to computerize (even with natural language questions), and the logical reasoning somewhere in between.
But I do not know! How long do we have before Watson’s progenitors can ace the exam? (I would hope that the answer is “never.” I do not think that it is.)
And in a slightly-less science-fictional vein, how good does the test appear as a metric of a student’s ability-to-benefit from a specialized high school? One problem is that the scoring is bizarrely non-linear: it would be better to do very high in one section and poorly in another than to do well in both. In practice, that weighs the math section over the verbal ones.
The article about the test at the link quotes experts as unanimously warning against the use of only a test in admissions. Boston Latin, for example, does not rely only on a test; nor do any elite universities. I am still unclear as to why so many of my high school compatriots argued that admissions requirements could not be changed.