Summer Reflections: Academic Dishonesty: cheat-resistant questions

This past semester (Spring 2021), I taught entirely online classes because of COVID-19 restrictions. In general, I really enjoyed and grew professionally from this experience. One drawback of any virtual course, though, is deciding whether and how to assess students using tests/exams. I chose to use online, untimed, un-proctored exams, so I knew that I should devise cheat-resistant exam questions that simultaneously help me easily detect cheating. This post describes my course design and exam design, including the types of questions I created that were successful at helping me spot cheating.

Course Design

My philosophy was heavily influenced by my professional development training in online course design as well as messaging from my campus and from my students. The argument, especially during a pandemic, was for providing flexibility. I experienced a lot of pushback on the idea of using remote proctoring services due to concerns about privacy and equity. Also, in part because of the potentially low-income and rural student population here, there were acute concerns about availability and stability of internet access and whether some students have safe and quiet places in which to complete timed exams.

I decided to assess my undergraduate course in genetics, with about 70 upper-division biology majors, with four midterm exams and a final exam. I also provided lots of lower-stakes assignments and exercises, so that the final grade would comprise:

30% Attendance and Participation (including asynchronous options for participation)
20% Exercises (homework, problem sets, etc.)
30% Midterm exams (the top three exam scores are used; the lowest score is dropped)

20% Final exam

Exam Design 

Thus, exams made up half of the student grade. It is worth noting here that my grading scale is atypical:

100-80% = A
80-60% = B
60-40% = C
40-20% = D
0-20% = F

I've written on the rationale for using this scale before. Briefly, I align all of the questions on my exams (and the points available for each) with Bloom's taxonomy, so that roughly 20% of points available on exams are lower-level Bloom's type work (like multiple-choice), and then 20% a bit more cognitively difficult, and so on. Thus, only students who are able to demonstrate competence at all levels (including the highest level, "Create") will be able to earn an A.

The reason I spend so much effort crafting these exams, and using this Bloom's Grading structure, is that this approach helps produce cheat-resistant exams. Students may easily be able to cheat on the lower Bloom's activities, like fill in the blank, true/false, and matching; with my grading scale, that might only earn them, at best, a C grade. Further, that's the scenario if I don't catch them cheating; if I do, then the grade is even lower. When students attempt the higher-level Bloom's questions that require them, for example, to analyze a dataset and then justify their answer, they necessarily produce unique responses. This makes it easy for the instructor to spot plagiarism.

Ultimately, designing exam questions to prevent cheating (or at least to make it easy to spot cheating!) does take effort. There is no perfect solution to writing cheat-proof exams, but you can improve their cheat-resistance.

There is a clear and direct trade-off between how cheatable an exam is and how much effort the instructor puts into creating the questions and into grading the responses.

Attempting to be an understanding and supportive instructor, this past semester I made all of the midterms and the final open-internet and asynchronous. Frankly, for an online course, and because I didn't want to use online proctoring systems, there's no practical way to prevent students from using all of the resources available to them. Plus, I've always allowed open-note exams, and I found no reason to change that policy. In terms of the amount of time given, I typically published the exams three days before they were due.

With this framework, students clearly have lots of opportunity to cheat by collaborating with each other, which was the one thing I specifically prohibit, including in this policy on the front page of each exam:

You may use all existing resources at your disposal (e.g. notes, course videos, resources found on the internet) to respond to these questions. However, you are not allowed to communicate with anybody else about this assignment (including, but not limited to: other students currently in this class, prior students, family, friends, or strangers who might respond to online discussion board/forum inquiries). By submitting your responses to this assessment, you certify that you agree with the statement: "I have done my own work and have neither given nor received unauthorized assistance on this work."

I didn't change the length or difficulty of the exams, relative to pre-COVID semesters when these would have been fifty-minute, in-person exams that I proctored. And, as always, I included some high point value higher-Bloom's questions that required paragraph-style writing and, often, opinions (e.g. "State an organism that you think is genetically modified, create your own definition of what it means to be a genetically modified organism, and explain how that organism meets your definition.") I also routinely require students to provide brief written justifications to their answers.

More Cheat-Resistant Questions

I have identified two related strategies for creating exam questions. These strategies can help you easily detect cheating. If you decide to explicitly tell your students in advance about this approach, then it might also help dissuade them from trying to cheat, too!

Strategy 1: create questions that have multiple correct and incorrect answers

The "Genetically modified organism" question above is a good representative of this approach. By asking a student to name one organism (and there are thousands upon thousands of species in the world), it is relatively unlikely that multiple students, by chance (i.e. without working together), will select the same organism. If they do, then that isn't itself proof of cheating, but it might suggest that you more closely scrutinize their written responses to look for other similarities.

Another example of this strategy is, in a genetics class, to ask each student to create the sequence of a mature mRNA molecule that a ribosome would translate into the amino acid sequence: MAYDAY* There are 128 different correct answers to this question. There is a tradeoff between how long the sequence is and how easy it is to score. In practice, picking five or six amino acids creates enough different correct answers and still makes it easy to score. The point is: multiple students submitting the exact same answer, of the 128 correct ones, would be highly unlikely and would be evidence for collaboration (cheating).

In yet another example, I provide students with a pedigree:

and then ask them, "Ignoring the dashed symbols, list all of the possible inheritance patterns." There are six patterns we learn about in class: autosomal dominant, autosomal recessive, X-linked dominant, X-linked recessive, Y-linked, and cytoplasmic. So, there are lots of possible combinations of those six types, but only one combination is correct. If multiple students submit the same incorrect combination, that could be evidence of cheating.

Next, I ask them to choose one of those possible inheritance patterns. Then, "Assume that the inheritance pattern you chose will exactly follow Mendelian inheritance patterns. Predict which (if any) the eight new children (dashed symbols) in generation IV will inherit this disease."

Depending on the inheritance pattern they choose, there are multiple correct answers and multiple incorrect answers. This is an easy question type to grade, because the correct answer patterns are simple to spot. For example, if the student said this was autosomal dominant, then half (five) of the ten shapes in generation IV should be filled in. However, the choice of which three additional shapes to fill in is up to each individual student. So, any similar patterns (either of correct OR incorrect answers) can suggest the possibility of student collaboration.

It is important to note, though, that there is always a chance (however small) that two students independently arrived at the same answer. So, it is worth diplomatically approaching accusations of cheating even with this sort of evidence.

Critically, solutions to these questions also are not Google-able: these are unique questions that don't have one correct answer.

Strategy 2: add unique phrases to help you web search your question

I wish I didn't need to employ this approach for virtual/asynchronous exams, but it can be very useful! The idea is simple: if you want to know if your students are posting your exam materials on the web (e.g. on a discussion forum), then efficiency demands that you be able quickly to sift through Google text search results.

When an exam question is written using common words, then there will be many results from a web search…and how many pages of search results are you willing to scroll through? Instead, I now invent scientific names of organisms (like: Albaratius torogonii), or create a unique "sample identifier" (like: "In the pedigree below, with database ID x456gh84i…"). There are currently no Google results for that scientific name and for that identifier, so if I include them in exam questions and students post the question online somewhere indexed by search engines, it is very easy to locate those materials.

This approach has been very successful in helping me find exam questions that were posted to Chegg…and dealing with Chegg will be the subject of an upcoming post!

If you have additional (and ideally no-cost) strategies you have found to be successful at preventing cheating and/or detecting it, please leave a comment below!