Teaching Tip #2 Three Quick Tips: Writing Objective Test Questions

(By me, published September 25 via Office of Professional Development at Oakton)

It's already a month into the semester (gah! How did that happen?!) and so you are probably working on or will have just handed out at least one big written exam, if that's a thing you do in your course. If you have a lot of students, you likely want to speed up grading if possible, and you might have opted to include some matching, true/false or multiple choice/single answer (MCSA) questions.

Educational research shows that the most reliable testing using these types of questions includes as many questions as possible. You also want to minimize the number of questions that will benefit poor students who know how to game the system or questions that slow down or penalize good students who truly know the material.

In my upcoming faculty seminar "How to write better assessments" we'll be looking at all sorts of ways to maximize the quality of your questions as well as how to make grading of essays and performances as easy as possible. For now here are a few teasers to get you started.

Matching

Example: Which of the following two silly matching questions can you do more quickly?

Question 1:

Matching: Put the best answer in the blank provided. Answers are used once or not at all.

 

____1. donkey

____2. cat

____3. snake

____4. polar bear

A. Covered in fur, chases mice, worshipped by Egyptians and YouTube viewers

B. covered in scales, has hidden ears, often the movie villain

C. an excellent swimmer, lives in sewers, misunderstood, trains cartoon ninja turtles

D. big soulful eyes, can be ridden in the Grand Canyon, might be Pinocchio

E. lives in Arctic, hides in snow eating a marshmallow, drinks Coca-Cola

Question 2.

Matching: Put the best answer in the blank provided. Answers are used once or not at all.

 

_____1. made of only one cell, can only be seen with a microscope, found in hot tubs

_____2. is green and thin, looks like it is praying, has an exoskeleton, the female eats the male

_____3. pushes poop balls around, looks like a scarab, world's strongest creature but no superhero is named after it

_____4. says "cluck cluck", has wings, tastes like everything.

A. amoeba

B. chicken

C. dung beetle

D. praying mantis

E. salmon

If you use Matching to have students match definitions or traits with a list of terms, names or other short identifiers, put the list with the shortest phrases or terms on the right in alphabetical order.  The good student will recognize your definition quickly, know the name or scan the list, find it and move on. If you do it the other way around, you are unnecessarily slowing down the students who studied.   

Note also you always give at least one extra option in the answer options column.  This avoids double jeopardy as well as process of elimination.

True/False Questions

Try not to use these; your students have a 50% chance of guessing correctly and that means that to make a reliable quiz or exam you’ll need a LOT of questions. 

 If you have no option but to use True/False (because you wrote it in your syllabus already this semester or some such) then:

  • Ask students to correct the false statements so you know they knew why the statement was false.

  • Never, ever use a statement copy/pasted from your handout, video or reading.  Students can recognize a phrase or sentence but still have no idea what it actually means. 

  • Students know that you'll statistically have more true than false statements, so if they get stuck on a couple they'll fill in accordingly.  Have a few more false statements than true to push back on that technique.  Also, don't have an alternating pattern of True and False statements; that's another pattern students watch for.

Multiple Choice, Single Answer (MCSA)

Example:

Multiple Choice. Select the best answer and write the corresponding letter in the blank provided.

 

____3. Last Sunday, Linnea Boyev ate which of the following?

A. A large drive through container of Baja macaroni and cheese

B. Seven Blow Pops

C. Thirty-four ounces of Diet Cherry Pepsi

D. Two bananas

E. All of the above

Of course, the answer is E. All of the Above.  You don't need to know anything about me to guess that.  On multiple choice exams, students know that if "All of the Above" or "None of the Above" appear suddenly, that will be the correct answer about two-thirds of the time.  Students are so trained to look for this that you can trick good students into answering “All of the Above” even on easy questions.

How to fix this?  Don't use All of the Above or None of the Above unless it is an option for every single question. 

 If you really want multiple answer type questions, avoid the complicated alphabet soup depicted in the following question:

 

____4. Which of the following were members of The Beatles? (Select the best answer; put the corresponding letter in the blank provided.)

A. John

B. Paul

C. Ringo

D. Steve

F. A and B

G. B and C

H. A, B and C

I. All of the Above

J. None of the Above

That sort of nightmare option array just slows everyone down.  Remember, if you want to use multiple choice format, your target is to ask more questions, not just a few unnecessarily hard ones. 

Multiple choice, multiple answer (MCMA) questions on the other hand, especially those where the student only gets credit if every blank is correctly filled in or left blank, are much more tricky. MCMA questions definitely slow everyone down.  Use them sparingly.  Students cannot use process of elimination.  On the positive side, studies show students will study much more thoroughly for MCMA than for MCSA (single answer) questions.  On the negative side you will end up asking fewer questions if you have limited exam time.

For example, a multiple choice, multiple answer (MCMA) Beatles question could have been written:

 

5. Which of the following were members of The Beatles?

(Select all that apply. There are between 1 and 4 correct answers.)

 ___John

 ___Paul

 ___Ringo

 ___Steve

Or, for those of you born after 1990 ("The Beatles" was a music band your parents liked), here's another animal identification question:

 

6. Which of the following is/are characteristic(s) of a polar bear?

(Select all that apply. There are between 1 and 4 correct answers.)

 ___has white fur

 ___eats baby seals

 ___has an exoskeleton

 ___lives in the Arctic

But note you only assessed polar bear knowledge in this question that is essentially four true/false questions.  In the same amount of time you could have asked four single answer questions about four different animals that good students would have breezed through.

 For more pro tips and fun with exams (hey, we don't have to take them!), join me for my faculty seminar "Writing Better Assessments" at 4pm on Wednesdays starting October 2 this semester.  [Editor- Offer no longer valid]

 ...and remember, when in doubt, always guess B.  (The "C" thing is a myth!)

Some links:

Teaching Tip #1 Three Gamification Tips You Can Use Today

(A slightly longer form of this article appeared as a Teaching Tip for Oakton on September 3, 2019 as a form of publicity for my September gamification professional development workshop)

If you've ever used a device, app or program that tracked your progress, awarded you a badge, or ranked you on a leaderboard, you've experienced a form of gamification. Gamification is defined as the application of game concepts to non-game situations. It's used widely in advertising and business to engage customers and employees, and we can use it to draw our students' attention away from their smart phones and back into the classroom.

The good news is that gamifying your course doesn't have to be complicated. You can do it easily through D2L/Brightspace and with a little planning you can gamify face-to-face lectures and discussions as well.

Here are few simple common elements of gamification theory you can start using right away to motivate students and to teach them not only content but also the benefits of good study practices.

1. Rewards/Achievements: What makes Fitbit and Candy Crush so addicting? It's the steady stream of intangible rewards like colorful buttons or banners that read "Great Job!" or "You got this!" that stimulate the pleasure centers of our brains. D2L has an incorporated "Awards" system with a wide variety of pre-made badges you can use. I've listed some YouTube links below that walk you through the steps of using badges. You can award badges for finishing readings or completing assignments; you can even make a "You signed up for the course!" that every student will get when they first sign on. In a face-to-face classroom consider handing out stickers or tokens. The receiving student can then turn those in at the end of class towards extra credit, or toward the overall points of their team? After all, "50 points for Gryffindor!" was very motivating to Harry Potter.

2. No stakes: Most folks play games to relax. We all have students that get overly anxious when it comes to grades. So some of the gamified tasks should not heavily graded or worth only a few points (pedagogically these are called "formative assignments"). An easy way to do this is online is with practice quizzes or tests on D2L. (And wouldn't you rather have students do practice tasks for badges than turn in homework you'd have to manually grade?) There are also multiple outside free sources you can link to (like Quizlet flashcard sets, Google's self-driven Peardeck worksheets or video platforms) that you can track as well.

3. Instant feedback: This is very important. In the same way a puppy learns a new trick, humans learn best if they know immediately whether if they did a task correctly or not. This is easy in face-to-face classes as you can tell students right away if they told you a correct or valid answer in discussion. Online, those hints or answers need to be on those practice quizzes right after the student takes them. Think about how those Facebook quizzes work online... not only does the taker get feedback right away, but they can show off to their friends. You can do the same thing with badges on D2L.

Other gamification concepts to consider include:

  • Embracing Failure as the road to "Leveling Up" (or as we used to call it: practice)

  • Multiple chances to gain badges/achievements

  • Teamwork

  • Short- and Long-term Goals

  • Short and Long-term Competition

  • Student/Player Agency

  • Visible Progress

  • ...and more

Links for D2L awards and badges videos:

Further Reading: