The College Board’s new method for raising AP scores

The organization released a report in July explaining the changes, and the transformation is radical. Between 2022 and 2026, approximately 1 million more AP Exams will receive scores of 3 or higher.

The College Board has released a preliminary explanation of what I have called the Great Recalibration of AP Exams. The report confirms that the central thesis of my article was correct: Hundreds of thousands of AP scores have been intentionally “recalibrated” upward since 2022. While I avoided sensational headlines about score “inflation,” I called the increase in scores a “radical transformation” of the AP program. But I had no idea just how radical the transformation actually was until the College Board released a report explaining the changes late last month.

Apparently, the College Board has not just been raising AP scores since 2022, it has completely reinvented the methodology it uses to assign AP scores. And this methodology has been designed to achieve a very specific result. It ensures that virtually all AP Exams have “the same 60–80 percent success rates” (i.e., AP scores of 3 or higher). This methodology only produces this pre-determined outcome. For most AP subjects, that means the scores stay the same, but for fourteen AP subjects (including many of the most popular subjects), it produces massive increases in scores. Whatever this methodology is, it never results in lower AP scores.

The new methodology, which the College Board calls “evidence based standard setting (EBSS),” was nowhere to be found on the College Board’s website until very recently, even though the report claims it has been used for the last three AP Exam seasons. As recently as May of this year, the pre-2022 scoring methodology was presented on the College Board’s website to explain how AP scores are assigned. In other words, the College Board’s public explanation of AP scoring has been inaccurate for the past three years. This is important because that page is virtually the only public source for this information, apart from some presentations at AP conferences and events.

How EBSS works

This new EBSS methodology has added layers of complexity to an already opaque process. I am neither a psychometrician nor a statistician, so I am not qualified to comment on the 2013 study cited in the College Board report as the basis for this framework. But the version of EBSS methodology in the College Board’s report seems to connect all kinds of disparate data into a single framework. The report does not explain how all these data forms a coherent methodology, so we can only assess these individual data points on their own terms.

The College Board’s report insists that the EBSS process “is especially well-suited for ensuring that AP standards and scores are not tugged higher by the well-documented increases in college grades over the past thirty years.” That seems like good news: The “inflation” of college grades is not causing an “inflation” of AP scores. But then this table appears in the College Board’s aforementioned report (the source of all five tables and figures below), explicitly comparing college grades to AP scores with no explanation of how AP scores are not being influenced by higher college grades:

Then the College Board presents two more tables explicitly comparing AP scores to college grades. I don’t understand. Are college grades used to help determine AP Exam scores or not? How is college grade “inflation” not “inflating” AP scores?

The EBSS methodology goes beyond comparing AP scores and college grades. Incredibly, the methodology uses tenth-grade PSAT scores to help compare college history classes to the AP U.S. History Exam. Besides the College Board’s own warning “not to ‘overuse’ test results” beyond their specific purpose, the connection between tenth-grade PSAT scores and college course performance is tenuous at best. And look at the column on the left, where college grades are listed with AP scores with no explanation of how AP scores “are not tugged higher” by college grades:

The EBSS methodology sometimes incorporates data that aren’t really data. Everyone who has been to high school and college knows that high school students spend more time in the classroom than college students do. Do we really need a whole bar graph to tell us this? This has been true for the entire history of the Advanced Placement program, so why would scores be increasing now? This graph also reminds us just how different college courses and AP courses actually are: The instructional hours are different, the pacing is different, the assignments are different. How does this graph help account for a 24 percent increase in AP scores of 3 or higher in a single year?

“Easier” rubrics

The report includes a section about “easier” AP History rubrics. Apparently, there were so many scores clustered at the lowest end of the rubric that they had to lower the bar to create a wider distribution: “If all points on the rubric are equally difficult to obtain, the scoring process will not generate as much data about students at the novice and intermediate levels of performance as it will about the most advanced students.” This is a perfectly valid reason for changing the rubrics, but it raises a legitimate question about why so many students were clustered at the lowest end of the scoring scale in the first place.

In this section, we also learn that the “complexity” point on AP History rubrics “provided no measurement value” because it “was rarely used by graders.” This of course raises questions about the similar “sophistication” point on AP English rubrics, which only about 8 percent of students earn. By this reasoning, the sophistication point should be simplified, too. Is the future of Advanced Placement one in which complexity is less complex and sophistication less sophisticated?

The future of AP scores

The College Board should be commended for releasing this report. It provides much more insight into the process than has been previously revealed. But AP teachers have rightly been confused and frustrated by the lack of transparency until this point. Why are we learning about this methodology for the first time three years into the process?

And why has the process been dragged out over several years? The College Board presented data in 2021 that showed the need to raise AP English Language scores, yet that exam has not been recalibrated. This report has also confirmed that five more AP subjects (AP English Language, AP Environmental Science, AP Human Geography, AP Latin, and AP Physics 1) will also be recalibrated according to the EBSS methodology. We do not know when they will be adjusted, but at least we have clarity about what to expect in the coming years.

The future of AP scores is now becoming clearer. By 2025, the Advanced Placement program will look very different than it did just a few years ago. We now know that nearly all AP Exams will be digital in 2025. We know the answer to Chester Finn’s question “Are AP Exams getting easier?” The answer is yes. Between 2022 and 2026, approximately 1 million more AP Exams will receive scores of 3 or higher as a result of the College Board’s new method for raising AP scores. As a teacher who wants students to succeed and earn college credits, I welcome the change. As an observer of the standardized testing space for the past two decades, I am amazed that it has taken three years for us to learn anything about this process.

Editor’s note: This was first published by Marco Learning.