|
Summer 2010
>>>Click here to download a printable copy.
Click here to visit the ICE News archives.
E-mail a friend!
Headline Articles
What is Your Certification Program’s Pass/Fail Rate?
Bhaskar Dawadi, Ph.D., Psychometric Program Director, Schroeder Measurement Technologies, Inc. (SMT)
One of the main purposes of licensure or certification is to protect the public. Organizations that offer certification and licensure examinations have the responsibility to assure the public that those who are certified or licensed have acquired an adequate level of knowledge and skill to practice competently in a particular field. One way to ensure that candidates entering into a profession are competent to practice in the field is by taking a well developed examination. When an exam is offered, some candidates will pass the exam and some will fail. A commonly asked question in the certification business is, “What percentage of candidates taking an exam should pass?” What is an acceptable pass/fail rate for a certification examination?
Is There a Standard?
There are no set standards within the field or body of literature indicating what percentage of candidates should pass a particular certification exam. In a certification industry survey conducted by the Institute of Credentialing Excellence (ICE), the average passing percentage for first-time test takers was 73 percent.1 The same survey reported an average passing percentage of 66 and 74 percent for first-time test takers in Business and Administrative Services and Health Care related certification programs respectively. However, since a certification examination is a criterion- referenced test in which an individual’s test score is referenced to a well defined criteria or domains of behaviors, candidates who are competent in those criteria or candidates who are fully knowledgeable in the identified domain of behaviors should pass, even if that is 100 percent of test takers. If candidates are not competent in those criteria or not knowledgeable in the domains of behavior, 100 percent of candidates should fail the exam, theoretically. If candidates fail the exam, the first thing that comes to mind is that they were not well prepared. However, in reality, if the majority of candidates fail the test, it cannot be blamed solely on candidate preparation.
Candidates who are interested in getting certified are fully aware that certification and licensing examinations are high stakes, meaning they are fully aware that their livelihood depends on whether or not they get certified. Many candidates select the area of practice they want to be in or the career they would like to develop with careful consideration. That is why those who prepare to sit for a certification examination spend a considerable amount of time and money in planning and preparing for the exam. Under these circumstances, when candidates sit for the exam and the majority fails, it can be surmised that it is not simply the candidates’ fault; there could be something else influencing the candidates’ performance on the examination.
Exam Validation
The organization offering the examination has a responsibility to review its examination-related policies and the testing program to make certain that the organization is doing all it can to help candidates succeed, and to make sure that competent candidates pass the exam and incompetent candidates fail the exam. How candidates perform on an exam is influenced by many factors. Examples include eligibility requirements, domains of behavior identified through a valid job analysis, clarity and comprehensiveness of the exam blue print, ease of candidate access to the exam blueprint, availability of study materials, quality of test items, severity on cut score, and testing time allocated.
One of the most important factors for the organization to consider is the candidate eligibility requirements. The organization should particularly review the scope of the eligibility requirements if all domains of knowledge being tested are covered by the requirements. For example, certain domains of knowledge identified by the job analysis for a particular certification program are taught only in undergraduate courses. However, if the minimum eligibility requirement to write the certification exam is only an associate degree, candidates may not test well because these candidates might not have been familiar with the content being tested.
The tasks to be performed and the knowledge and skills necessary to perform those tasks are identified by conducting a detailed job analysis. Defining the level of eligibility requirements necessary to perform those tasks identified by the job analysis can be challenging. What is the requisite level of candidate education in order to perform those tasks competently? Is it necessary for candidates to have a certain number of years of work experience before they come to take the exam? Defining eligibility requirements in a way where the public is protected by only certifying competent professionals in the field, but not impeding competent candidates from being eligible by overly-restrictive eligibility requirements is critical. Eligibility requirements should not be set as an unreasonable barrier to entry into a profession. The Joint Standards for Educational and Psychological Testing (AERA, APA, and NCME, 1999) state:
Standard 14.17
The level of performance required for passing a credentialing test should depend on the knowledge and skills necessary for acceptable performance in the occupation or profession and should not be adjusted to regulate the number or proportion of persons passing the test.
The cut-score is another factor that has a large influence on whether candidates pass or fail an examination. In a certification and licensing exam, the cut-score is determined by a panel of subject matter experts (SMEs). These SMEs should be knowledgeable about the target population for the examination and should understand the content material being covered by the examination. SMEs must set the passing standard by being objective, fair and unbiased to the minimally competent group of candidates. It is easy for experts to forget what knowledge and skills they possessed when they first entered the field. Experts have many years of experience that have not only provided new knowledge and skills, but also improved upon others. It is important that SMEs set cut-scores based upon the eligibility requirements with entry-level candidates in mind, which may take some reminding and careful thought.
Moreover, the organization should actively disseminate information about the exam, such as the exam blue print and the domain weights, informing candidates of the relative importance of each area and also directing them to the proper study materials.
The quality of items will also impact how candidates perform on the examination. It is always a good idea to pre-test new items by including these items in an actual examination. Statistical analysis conducted using the data received from a group of candidates taking the actual examination could help improve the quality of the items. Next, the time allotted to take the examination could have an impact on candidates’ performance on the examination. Do candidates have enough time to respond to all items presented on the test? Candidates should be given enough time to complete the exam; at least 90 percent of candidates should complete the exam in the allotted amount of time. Otherwise, test speediness can occur (candidates run out of time and are not given the opportunity to answer all the questions), which is a threat to the validity of the exam. Lastly, the organization should monitor the performance of candidates for an extended period of time to assure that examinations are performing as expected.
When the organization offering the exam considers the aforementioned activities, it will help ensure that competent candidates succeed on the certification exam. Most importantly, it will help protect the safety and well-being of the public and will provide a fair pass rate on the exam.
Please send comments regarding this article to the author at bdawadi@smttest.com.
Dr. Bhaskar Dawadi is the Psychometric Program Director for Schroeder Measurement Technologies, Inc. Dr. Dawadi is fluent in four languages, holds three Masters Degrees and a Ph.D. in Measurement and Statistics from Florida State University. Prior to joining SMT, Dr. Dawadi’s experience included the Nuclear Medicine Technology Certification Board (NMTCB), the Georgia Examination Development and Testing Unit and the Florida Department of Business and Professional Regulation (DBPR).
Footnotes
“Certification Industry Operating Ratios and Benchmarking Statistics” 1st edition. Institute for Credentialing Excellence (ICE). Washington, DC.
ICE Academy to Include Marketing Course
ICE will be offering Credentialing Marketing for Non-Marketing Professionals, a new course designed to help credentialing professionals effectively communicate the value of their programs to the marketplace. Geared for credentialing professionals who are responsible for marketing their own programs, the course will help participants understand the basics behind identifying a unique value proposition, developing a strategic and tactical marketing plan, and measuring success. The 90-day course’s blended-learning approach combines real-time Webinars, online learning modules, and coursework to provide optimal learning opportunities. Attendees will be provided practical templates and resources, an interactive workbook, and on-demand coursework.
Courses are scheduled to begin in fall of 2010. ICE members receive discounted registration rates. Watch for registration details later this summer!
Credentialing Marketing for Non-Marketing Professionals is part of the ICE Academy family of educational offerings, designed to help credentialing organizations deliver high quality programming and successfully manage administrative operations.
ICE News is published by the Institute for Credentialing Excellence, 2025 M Street, NW, Suite 800, Washington, DC 20036. For more information call (202) 267-1165 (202) 267-1165 or email info@credentialingexcellence.org. Jim Kendzel , Executive Director.
|