CAP Assessments: Determining the Passing Score

“Why do I need to get 21 questions correct to pass this CAP assessment when I only needed 20 correct questions on the last CAP assessment?”  This is a question that we hear frequently from our Diplomates.

Unlike tests that you took during school, which were norm-referenced assessments, certification or licensure exams are criterion-referenced assessments. Norm-referenced assessments compare the test-taker’s performance to a peer group and provide a comparative ranking, separating the high achievers from the low achievers. Criterion-referenced assessments determine mastery of a specific subject or standard, rather than determining how the test taker compares to others.

The passing score of a criterion-referenced assessment requires a psychometric process known as standard setting. This process determines the score at which a test-taker is deemed to have “met the standard.” It is independent of the other test-takers as it solely references the content of the test.

The CAP assessments form the basis of ABO Diplomate recertification and are therefore considered criterion-referenced exams. It is not legally defensible to arbitrarily pick a cut score (such as 70%) for a criterion-referenced exam. Instead, each CAP assessment undergoes a standard setting process using a modified-Angoff method, which is the same methodology used to determine the passing score for the board certification examination. A panel of ABO Diplomates reviews the assessment and determines the difficulty of each test question. Those individual judgments are then combined to calculate the pass/fail cut score. An assessment with a higher cut score, such as 21/25, is considered to have an overall difficulty that is easier than an assessment with a lower cut score, such as 19/25.

The pass/fail cut score is always indicated in the instructions for the assessment. Be sure to check instruction #5 before beginning any CAP assessment!