Creating Multiple Choice Tests
Below are ten important rules for anyone who has to write multiple choice questions. In the event you have any other tips, please feel free to tell us about them.
Ten Essential Rules
- Rule: Understanding of test and ability to think critically – rather than simply relying on recall
Questions of the multiple choice variety often get criticized for only testing the knowledge of test-takers at a superficial level. However, it is possible to go deeper than this by requiring candidate’s to evaluate various situations, give their interpretation of facts, draw inferences, forecast results, and explain causes and effects.
- Rule: Keep your wording precise and the structure of your sentences simple
The structure of your questions should be sufficiently simple to make them easy for test-takers to understand. You should additionally aim to make the choice of words in your questions as accurate as you possibly can. It is possible for some words to have several meanings with much dependent on context and how words are used in a colloquial sense.
- Rule: The majority of a question’s words should be placed in the stem of the question
In the event you want to use a question’s stem, instead of a whole question, make sure that you place the majority of the words in the question’s stem. This approach gives scope for making the choice of answers shorter and, therefore, easier to read and not so confusing.
- Rule: Distractors should be plausible
It is important that all your choices of incorrect answers are entirely reasonable. It can be quite difficult to achieve this, but you should avoid putting in obvious distractors (sometimes spelt “distracters”) because these can take away from the credibility of a test. In the event you really get stuck, you can seek advice from your trusted SME.
- Rule: Try to keep answer options a similar length
It can be very difficult to keep answer options the same length, but seasoned test-takers are likely to use the length of answers to deduce which one is correct. In many cases, the correct answer is the longest one. Some expert test writers use two long and two short answers when they cannot think of four answer options of a similar length.
- Rule: Double negatives should be avoided
There should be no surprise here. Combinations of the following words should not be used in any one question: no, not, nor, or the prefix –un, and so on. Consider this example: “Of the options below, which would NOT be unhelpful in a workplace?” Turn the option around to its positive version, e.g., “Of the options below, which would be helpful in a workplace?”
- Rule: Present right answers in mixed order
Double check that the majority of right answers are not in the same positions e.g. as options “a” and “b.” This is a frequent mistake. So make sure you present the right answers in random fashion and do not allow them to fall into a detectable pattern. When you have written your test, go back over it and, if necessary, re-arrange the order of the right answers.
- Rule: The number of answer options should be kept consistent
Have you ever tried convincing a shrewd SME that they cannot have answer options that go up to “i” in some of the questions and only to “c” in others? This issue has a lot to do with user interface. Having a consistent number of answer options in these types of questions can help test-takers to know what they can expect. Despite different studies on the matter, none has agreed as to whether the best option is 3, 4, or 6 choices. However, some test-writers think four is a fair option.
- Rule: Try not to trick test candidates
Even though many may have faults, the reason for tests is to measure the knowledge of test-takers. So, try not to include any questions or any answer choices that might trick those participating in a test. If there are any questions or possible answer options that could be interpreted in more than one way or if there is a very subtle difference between various options, then it is best to try and rewrite these.
- Rule: Be Careful When Using “All” or “None” in Answer Options
You will often see “All of the above” or “None of the above” as answer options in multiple-choice questions. However, a lot of test-writers do not like this rule because, while it is useful when they cannot think of any more distracters, it does not necessarily support the idea of effective learning. This is because the “All” option can be a none-too-subtle give-away if it is used inconsistently. Additionally, the “All” option can lead to a lot of guesswork in cases where a test-taker believes a couple of the answers to be correct. The disadvantage of the “None” option is primarily that it prevents an evaluator from knowing whether the test-taker really did know the right answer.