Step 1 Goes Generic

Assessing medical knowledge has been one of those bugaboos of medical education that has multiple purposes.  By the time a person gets to medical school, they have taken high stakes exams, from SATs, in my day College Board Achievement Tests, the Organic Chemistry final, almost certainly an MCAT once or twice.  All have numbers attached to them, designed in part to assess achievement but also to predict future potential, as these scores get submitted to others who don't really care how much math you really know but have to sort through who to select for limited entry slots.  Medical school Admission Committees skim off the success stories and regroup them into a new pool for new high stakes exams that never really end.  

The general sequence of standardized testing, not counting course examinations, includes Step 1 after the preclinical years, which for some schools tacks on the core clinical year, Step 2 during the senior year, Step 3 usually taken after internship, Board Certification after residency or fellowship, and Recertification which occurs typically at ten year intervals for most specialties.  We get pretty adept at taking them.  

Each ordeal has a number assigned to it.  The purpose of the score becomes less clear as the cohort of test takers becomes more exclusive.  Specialty certification exams, the final step, are functionally pass/fail.  Either you achieve certification or you don't, yet there is a score reported and subdivided by subject for the sole benefit of the applicant to know where some future attention to knowledge upgrades are best applied.  By Step 3, admission decisions have already been made so only a pass benefits the new physician, but the institution needs to know the caliber of those accepted into their programs and options for tweaking their curriculums.  By the time Step 2 is reported, residency match applications have been submitted.  There is a specialty score which may help the program rank its applicants but the scores benefit the medical school more than they benefit the student, especially in view of a 97% pass rate among first time takers. .

Step 1 may be the last test with serious ranking potential both for student and for school.  A recent decision to no longer report scores beyond pass/fail has created some controversy.  Good test takers feel cheated.  Program directors looking at hundreds of applicants from scores of schools liked the ease of lopping off the bottom 2/3 from further consideration.  Students whose test taking savvy fell below their grades, which are more a reflection of daily diligence, now have an advantage.  Is this a good thing?  Nobody knows yet.

In my era we had Part I of National Boards.  My school required a passing score to proceed to the clinical year.  Those who did not pass, about a dozen each year, and not necessarily those who struggled most with their courses, were assigned a remedial summer while the rest of us went to our first clinical rotation.  On repeat examination in September, our pass rate approached 100% and nobody's career was seriously impeded as much as their self-esteem was.  Scores went with our residency applications that could decide what to do with the results of preclinical years.  The biggest beneficiary was probably the school which could design its curriculum to maximize pass rates, assess the effectiveness of its pre-clinical faculty, and target students weak in a particular science for remediation in advance of the exam.  Knowing the historical pass rates also helped.  That is no more.  While in my day, we had scores that would be used later, the immediate utility was an overall Pass that punched the ticket into the clinical years.

As it goes pass/fail with no scores to be passed along either for residency screening or self-assessment, something beneficial is likely to be sacrificed.  Since the pass rates are the same as when I took them 40 years ago, there is still that element of anxiety among the test takers but it won't haunt them two years later.  They can focus on their classes.  The schools can focus on their curriculums without concern of their relative standings among schools.  But the residency program directors will need a different screening mechanism.  Likely those of the more prestigious schools will have a selection advantage that a diamond in the rough who squeaked by the competitive admissions process but thrived once on campus can no longer compensate.  Overall, I suspect people other than the program directors will eventually miss those Step 1 scores.