Scoring the Exams

There are two aspects to scoring the exams leading to the CHRP designation (the National Knowledge Exam® (NKE) and the National Professional Practice Assessment® (NPPA)):

1. Angoff panel to determine the cut-score and

2. Removing non-performing items in the final scoring stage.

Angoff Panel 

The October 2009 administration of the exams introduction of the use of Angoff panels to set the cut-score on the exam.  An Angoff panel is a method which makes use of the combined judgment of panel members to establish the probability that a candidate at the threshold of competence would be able to answer the question correctly.

The table below gives an example of what Angoff panel data looks like and how the cut-score is arrived at.  The data in the table are fictitious, and for this example represent the items on an NKE exam only. The process is the same for the NPPA, however there would be 60 items reviewed as opposed to the 150 in the example below.  After all panel members have reviewed all questions, the probabilities are averaged across panel members to arrive at an average probability for each item.  Summing the average probabilities for each question gives the proposed cut-score for the whole test.

In the above example, the cut-score for the NKE would be set at 99.  Expressed as a percent, the cut-score would be 66%.

Because the cut-score for the test is derived by adding the probabilities for each question, the cut-score will vary depending on the particular set of items that make up the test.  Each version of each exam will have its own cut-score.  An exam that is made up of somewhat more difficult questions will have a somewhat lower cut-score; an exam that is made up of somewhat easier questions will have a somewhat higher cut-score.

These Angoff panels are convened before each exam. 

Post-exam Review

The exams are scored using a two-pass process.  Although all test items are carefully written and selected, it happens that some test items do not perform as expected.  In a first pass, potentially flawed items are identified based on statistical criteria.  These items are reviewed and some may be discarded.  In a second pass, the final scores and cut-score are recalculated on the basis of the retained items.

The statistics calculated for each item include difficulty and discrimination indices for each option.  In addition, difficulty indices are calculated for candidates at varying levels of overall exam performance, as well as for each linguistic version of the exam.

All statistically flagged items are re-reviewed by the Exam Board in view of making a final decision as to the inclusion of the item.  There are various reasons why items may fail to perform as expected: items that inadvertently have no correct answer or more than one correct answer, items that are not at the appropriate level, or are ambiguous in some other way.  Sometimes, items are found to be mis-keyed, in such cases the item is re-keyed and the item statistics re-calculated.

Final Scoring

All items that the Exam Board identify as non-performing are deleted from the final scoring.  The Angoff panel information for the deleted item is also discarded.  In a second pass, the scores are recalculated omitting the discarded items.  This score is compared to the Angoff panel cut-score recommendation also omitting the discarded items.  Scores at or above the threshold are given a pass.

Starting in October 2012, the National Knowledge Exam (NKE) overall scores will be reported as scaled scores.  CCHRA scaled scores are created when the number of questions that candidates answer correctly is mathematically transformed.  Specifically, this transformation is an arithmetic conversion of the raw scores to a scale that ranges from 200 to 800 with a passing score of 500.  Scaled scoring is commonly used in similar standardized tests and does not alter the way the NKE measures candidates’ knowledge. 

Although the cut-score of 500 will not change from one session of the NKE to the next, the number of questions that have to be answered correctly to achieve a passing score can change. If a session of the exam is easier than the exam on which the passing score was set, more questions would have to be answered correctly to obtain a scaled score of 500. If the session of the exam is more difficult, fewer questions would have to be answered correctly to pass the exam. Thus, candidates are not unfairly rewarded because their test was easier nor unfairly penalized because it was more difficult.

Manual Re-scores

Manual re-scores are available for those who believe that there has been an error in the scoring of their exam. If you wish to have your exam manually re-scored, please send a written request along with payment of $50 for the NPPA or the NKE to:

     Attn: Professional Standards
              CCHRA
              130 Slater Street, Suite 1025
              Ottawa ON, K1P 5L7