In all seriousness, the volume and quality of peer-reviewed evidence in favor of three-option multiple-choice questions are astounding.

Instructional Design Based on Evidence

How many alternatives should there be in a multiple-choice question? Four. Obviously. There is one accurate response and three distractions. That concludes the story. Why are there four? So…take a look around! Four alternatives are just the correct number.

I huddled in a gloomy, firelit cave with other Gen X youngsters to learn my ABCs back when dinosaurs roamed the globe, and everyone wore flannel for some reason. Almost every test I took, as far as I recall, included four-option multiple-choice questions. Four! Later, my ID professors informed me that four alternatives were the correct number. As a young designer, the instructional standards I worked from also required four. Four! Even quiz templates included with popular eLearning development software is pre-programmed with four-option multiple-choice questions. Four!

But I’ll play along. Let’s look at the peer-reviewed study on the off-chance that I’m wrong about this.

Investigation into the Number of Options in Multiple Choice Assessment Questions

“The findings show that three-option MCQs save time and allow for more content and items to be covered in the test, boosting test validity and reliability.” (A Comparison of Three- and Four-Choice Multiple-Choice Questions)

“Students answered three-option MCQs 5 seconds faster on average than four- and five-option MCQs (36 seconds versus 41 seconds; p = 0.008). There were no statistically significant variations in item difficulty, discrimination, or test reliability.” (Reducing the Number of Options on MultipleChoice Questions: Response Time, Psychometrics, and Standardization)

“The study found that did not have a significant effect on mean test scores, mean test difficulty, or test discrimination indices, but did affect internal consistency reliability coefficient. The findings, in general, gave more evidence to support the usage of three options.” (The Psychometric Characteristics and the Number of Options in a Multiple-Choice Test Item)

According to a meta-analysis of 80 years of study on multiple-choice assessment creation and evaluation, three alternatives are the ideal number. (A MetaAnalysis of 80 Years of Research Shows That Three Options Are Optimal for Multiple-Choice Items)

Conclusions

So, what were we talking about all of a sudden? Yes, as I previously stated, three alternatives are the optimal number for multiple-choice questions. Sure, some old-timers still talk about four, but that was five minutes ago, you know?

In all seriousness, the volume and quality of peer-reviewed evidence in favor of three-option multiple-choice questions are extraordinary. I listed some studies above, but there were at least a dozen others that all supported the use of three-option questions. Based on all of this high-octane nerdery, we may confidently conclude:

• Three-option questions have the same reliability and validity as four- or five-option questions. (In fact, some research indicates that three-option questions are more reliable and valid than four- or five-option ones.)
• Three-option questions do not make a test easier to pass when compared to four- or five-option questions.
• Scores for three-option questions do not differ significantly from scores for four- or five-option questions
• Students can finish three-option questions faster than four- or five-option questions.

Implications for Practice

Three-option multiple-choice questions appear to be as excellent as, if not better than, four- or five-option questions in every area that a working ID may care about. Furthermore, because learners may complete three-option questions more rapidly, employing them will reduce sitting time by enhancing instructional efficiency.

But how much seat time can you realistically hope to save? Let’s go with the data from the study above as a general guide: around 5 seconds for each question. That’s a major issue for a psychometrician who creates standardized tests. It means that more questions can be answered in a given amount of time, increasing the test’s validity and reliability.

But how much of a difference will it make for the average ID developing workplace training? It is determined by the size of your audience. Assume you’ve designed a course with ten multiple-choice questions. Using three-option questions will save you approximately 50 seconds for each student. If 100 people take your course, you will save approximately 83 minutes of productivity. If 10,000 people complete your course, you will have saved your organization around 139 hours of seat time.

Even if you aren’t going to drastically cut sitting time, there is another compelling reason to use the three-option standard: development time. The research I presented above does not directly address this, but it seems to reason that constructing two distractors will take less time than creating three. Assume you have 5 minutes to build and validate a distractor. For a course with ten multiple-choice questions, switching to the three-item standard will save you 50 minutes in development time.

Three lives on!

I’m primarily interested in fresh developments. I’m not going back to retake old courses. I haven’t done the math, but I suspect that the additional development and administrative work required to rewrite and re-deploy old courses will be justified by seat time savings.