Writing Good Test Items
This post was written by Judith Wightman, Professional Development Fellow and faculty in the Psychology department.
One skill that most college instructors use on a regular basis is writing valid and reliable test items. These items may be based on any source material used in the discipline and can be used for a variety of purposes, including checks on completion of reading assignments, and formative and summative assessment. The quality of these items matters, both in terms of testing on what we think we’re teaching, and in terms of maintaining positive relationships with students. I recall the first class that I taught independently, and the first time I returned an exam that I wrote myself to students. The flood gates opened, and students shared with me a long list of complaints about the exam. One student actually passed me a note on the way out the door that said, “I’m sorry that happened to you.” For a few semesters after that, I shared “ground rules” with students before passing back their exams, but I also took a hard look at whether the hostile students made some legitimate points. I found that they did.
Since then, I have learned to write test items that are fair, that distinguish between students who know the material and those who don’t, that are challenging without being “tricky”, and that provide a reasonable proxy for the degree to which students have learned what I have attempted to teach. No question or exam is perfect, and everything is measured with error. But I’m better off now than I was in that first class, and my students are, too.
I am currently leading an item writing workshop to share with others the experiences I have gained in teaching and while working as an item writer for ACT. My one participant this semester is Greg Krawiec, a second-year faculty member in Hospitality Arts. While I hope to recruit a group of folks to participate the next time the series is offered, it has been fun to work with Greg on the kinds of items he wants to move to in the Hospitality courses he teaches. We have talked about best practices in writing multiple-choice, true/false, matching, and essay items, as well as Likert-scale items for opinion surveys. I wrote some items based on a textbook that Greg uses in his classes, which gave us a chance to talk about how to write items that reflect day-to-day challenges faced by workers in the hospitality industry.
All faculty are invited to participate in a future item writing series. Please contact me if you would like to see the series offered on a particular day and time, or if you would prefer one-on-one assistance.
About Judith Wightman
My degree is in industrial/organizational psychology, which is the study of psychology applied to the workplace. KCELT provides many opportunities for organization development and I’m excited to contribute to the work of the Professional Development Fellows team. I teach Introduction to Psychology, Developmental Psychology, Social Psychology, and Human Sexuality in face-to-face and online formats.