Authentic Open Book Assessments?

This semester I’ve heard a lot of discussion about test formats, cheating, and all things related to assessment integrity. These are all very valid discussions, and we are certainly facing some new circumstances in regards to assessing our students. One question that I didn’t worry about with my students was, “How do I keep them from looking at their textbook/notes while taking a test?” I didn’t worry about it, because it’s been years since I gave closed books tests (except when required by state/school regulations, of course). While I don’t claim to have all the answers, I’d like to explain my reasoning for this particular policy in my classroom.

The short answer is, I think open book/note tests are more authentic. By that, I mean that they more closely mimic the circumstances students will be expected to perform under in the real world. (As an aside, this is also my rational behind my no late work policy and exceptions, as well as a few other classroom procedures/policies.) I will admit that my work experience outside of education is more limited than most, but I have held several different jobs that had absolutely nothing to do with education. I have also held a variety of education-related positions, for a plethora of different schools/organizations, and at various levels. So while my work experience may be more limited than others’, it is far from non-existent. One thing that I have always experienced, and observed in the work experience of family and friends who aren’t educators, is that it is extremely rare that one is required to complete a task with nothing but the knowledge that exists in one’s head. Rather, what commonly occurs is that one is given a task to complete and an amount of time within which to competently complete it. Occasionally further parameters will be set, such as allowable or required materials or a budget, but rarely is one told that one may not use whatever resources are available. In short, your boss expects that you will use available resources to produce quality (or at least competent) results in a (hopefully) reasonable time frame.

I believe a test can mimic this real-world situation:

  • quality (competent) product = correct answers on an assessment
  • reasonable time frame = amount of time given to complete assessment
  • available resources = textbooks, notes, and even (dare I say it?) the internet

Of course to produce this quality product in a reasonable time frame, students will not (or at least should not) be able to look up every answer in a book, notes, or even Google. When I give open book/notes tests, I do not give unlimited time. I give a reasonable amount of time. If students do not have at least a basic level of knowledge in their heads, they will never be able to finish enough of the assessment to receive a passing grade. The more knowledge they have in their heads, the more time they’ll be able to spend transferring that knowledge to the paper, thus resulting in a higher grade.

Another factor that allows me to confidently assess students using this open book model is the type of assessments I give. It is rare that I pass out multiple choice, matching, or even fill-in-the-blank assessments. I prefer assessments that are, at minimum, short answer or essays. This means students have to express their knowledge in their own words, they can’t simply Google the question and write a one or two word response (or circle a single letter). Let’s take the listening course that I taught last semester as an example. I am required to assess students’ listening comprehension skills, grammar skills, and vocabulary skills. In order to do this I could use the standard test produced by the curriculum company that has vocabulary and definition matching questions, short listening passages with basic comprehension questions, and fill-in-the blank questions for grammar. Instead, I use TED Talks. I find a TED Talk that is related to the themes we’ve been studying, or is something that will be interesting to my students. I then design a series of comprehension questions that require different levels of responses. Students are required to listen to the TED Talk and write complete sentence answers for each question. With one TED Talk, and 7-10 questions, I’m able to assess all the required skills:

  • Listening Comprehension: some questions are factual in nature, others require evaluations or opinions, if students understand what the speaker said, they’ll be able to answer the questions. This last semester I had one student who had great answers to all of the questions, but the answers were all expressing something in direct opposition to the speaker’s main point. It was very obvious that the student had understood some of the speech, but not much.
  • Grammar: because all answers are written in complete sentences, I can easily see if students can express themselves using proper grammar or not. Since none of the questions are worth a single point, I’m able to take off a point, or more, for poor grammar, and give feedback to help the student improve.
  • Vocabulary: I will admit that it is rare that I find a TED Talk that uses most, or even many, of the specific vocabulary words we’ve discussed in our units. This would be a problem if I believed that knowing those specific words was the most important part of our vocabulary instruction. It’s not. I actually think vocabulary skills, such as context clues, are much more valuable than a specific list of words (before you start emailing and commenting with all the reasons why students need to learn specific words, please look at the vocabulary section of my blog. I do teach specific words, especially academic vocabulary, and I even test it.). I often include questions that ask students to define specific words/terms from the talk, or to give their opinion relating to them. This allows me to assess what I really want to know: can you understand, and communicate in relation to, the overall message, even if you don’t know every word.

My summative assessments were timed (my school uses Blackboard, which allows for timed tests), but they also had four formative assessments of this type, all of which were untimed. In my opinion this too is more authentic. It is rare that you have to give a big presentation to a client, release a new piece of software, or begin selling a new product without having some sort of practice with it first. The practice may be on a much smaller scale, or with a slightly different product, but you get the opportunity to do a trial run of some kind before the final release. This is why I prefer to not give summative assessments that do not mimic or contain pieces of formative assessments from the previous unit(s).

As I said in the beginning, I don’t claim to have all the answers. I’m not even claiming to have the best answer to this particular question! This is just the approach to assessment that I’ve developed/used over the last decade or so, and it seems to work well for me. Every time I’ve had the opportunity to explain it to an administrator or parent, he/she has affirmed the logic and ultimately supported my approach. Students have all been receptive as well (how much explanation I offer or give in response to questions depends on the age of the student, but I’ve used this method with students from third grade through college); and while I’m sure at the beginning they were just happy to get “easier” tests, I know by the end they really appreciated how much learning they had to demonstrate. It’s not a method that I would force onto anyone else, but it does work for me, and I encourage you to consider if, or how, it could fit with your own personal teaching style. Happy teaching, everyone!