WP15 What Is the Optimal Number of Distractors in Exam Items? Case Study

Multiple choice (MC) items that employ four or more options remain a popular format in standardized college entrance exams and employment selection tests despite the fact that most research supports the use of three-option MC items. The same is true with licensing and credentialing exams. A case study is provided that documents the Chartered Financial Analyst (CFA) exam’s transition to a three-option item format. The CFA program is the gold standard credential among investment professionals globally, with 124,000 CFA charterholders residing in 145 countries. This case study is intended to be of value to other testing programs considering a similar change with respect to understanding the motivation for change, the challenges faced with stakeholders and constituents, and the psychometric impacts of change that align with literature-based expectations.

Meet the Authors

Peter Mackey, CFA is Head, CFA Program and Examination Development at CFA Institute.  In that role, Peter coordinates the major activities supporting the CFA Program and manages the exam development work for all certification and certificate programs offered by CFA Institute.  Before coming to CFA Institute in September 2004, Peter spent five years at the U.S. Treasury Department as Deputy Director at the Office of District of Colombia Pensions where he managed three trust funds on behalf of the District’s defined benefit pension plans.  Peter’s experience also includes 12 years as Director of the Office of Investments for the Federal Retirement Thrift Investment Board, where he established a defined contribution retirement program for US government employees and directed an investment fund of more than US$80 billion on behalf of 2.4 million participants.  Peter has been involved (as a volunteer, consultant and staff member) in all aspects of practice analysis, curriculum development, exam design and development, exam grading and standard setting for the past 20 years.  He holds an MBA in finance from George Washington University and a BA in economics from Manhattan College.

Timothy Konold, PhD is Professor and Director of the Research, Statistics, and Evaluation program at the University of Virginia; and holds faculty affiliations with the Center for the Advanced Study of Teaching and Learning (CASTL) and the Virginia Education Science Training (VEST) program.  He has taught introductory and advanced graduate level courses in psychometric theory and advanced quantitative methods at the University of Virginia for the past 18 years.  He has served as the senior psychometric consultant for the Chartered Financial Analyst (CFA) global testing program for over ten years.  He has authored more than 100 peer-reviewed articles, book chapters, published tests, book/test reviews, and technical reports on topics that seek to infuse contemporary psychometric and quantitative methods into work that has direct implications for large scale test use as it pertains to construction, interpretation, classification and errors of measurement.

Member Price: FREE; Non-member Price: $50

You must log in to your account, become an ICE member or create a non-member account to register for and/or purchase this item.

If you receive ICE emails, you already have an account. Click 'Login to view purchase options' to login or retrieve your password using the "Username and/or Password Help" link at the bottom of the page.

Login to view purchase options