How to learn with practice questions

"Chew and pour" is the phrase, and anyone who has sat an exam in Ghana knows the routine it names: read the handout until the pages go soft, highlight what looks important, and then, the night before, go through a stack of past questions with the marking scheme open — not to answer them, but to rehearse the answers. In 2020, Amoako, Sharma and Danquah surveyed 800 students across KNUST, the University of Ghana, Cape Coast and Cape Coast Technical University about the habit they call memorise, reproduce and forget. The students knew what it was. They did it anyway, because it is what the timetable leaves room for.

This article is not going to tell you to stop using past questions. It is going to tell you that you are using them at the wrong end of the process. A question is not the check you run when the studying is finished. It is the studying. The experiments that established this are decades old, have been repeated in hundreds of classrooms, and every one of them reports the same second finding: the students taking part predicted the opposite.

What follows is what those experiments found, why the method works, the honest answer to "but I need the basics first", and a seven-rule method you can run on any course this week.

What happens when you test instead of reread

Three experiments carry most of the weight, and each is short enough to describe properly.

In 2006, Henry Roediger and Jeffrey Karpicke gave undergraduates a prose passage to learn, in Test-enhanced learning, Psychological Science 17(3). One group read it in four sessions of five minutes; by the end they had read it, on average, 14.2 times. Another group read it once and then spent the other three sessions writing down everything they could remember — no notes, no feedback, no second look. They read it 3.4 times. Five minutes after the last session, the rereaders were ahead, 83% of the passage recalled against 71%. One week later the order had reversed: the rereaders recalled 40%, the group that had tested itself 61%. Over the week the rereaders forgot 52% of what they had known; the tested group forgot 14%.

The detail that matters most is the one students never guess. Asked to rate how well they would remember the passage in a week, the rereaders were the more confident group: 4.8 on a seven-point scale against 4.0. The method that felt best was the one that worked worst.

Two years later the same pair ran an experiment that answers a question you have probably asked yourself: once I can recall something, can I stop practising it? In The critical importance of retrieval for learning, Science 319 (2008), students learned forty Swahili–English word pairs. Every group reached the same point — every pair recalled at least once — and every group predicted they would remember about half the list a week later. Then the groups diverged in how they treated a pair once it had been recalled. Where a recalled pair was dropped from further testing, a week later students remembered 36% of the list in one condition and 33% in another. Where recalled pairs stayed in the tests, students remembered about 80%. Whether the pairs were restudied made no measurable difference at all: the extra study trials, roughly eighty of them, bought nothing. The gap between the tested and untested groups was four standard deviations, and the two sets of scores did not overlap — the best student in a drop-from-testing group scored below the worst student in a keep-testing group.

The third experiment removed the last comfortable objection, which is that recalling a passage is fine for facts but not for understanding. In Retrieval practice produces more learning than elaborative studying with concept mapping, Science 331 (2011), Karpicke and Janell Blunt matched the study time exactly. One group of students built concept maps of a science text — the activity most study-skills advice holds up as the model of deep, active learning. The other group put the text away and wrote out what they could recall, twice. A week later, on a test that included inference questions requiring students to connect ideas the text had not connected for them, the retrieval group scored 0.67 to the concept-mappers' 0.45: about 50% better, an effect size of 1.5. A second experiment had 120 students do both, one method for each of two texts. Of the 120, 101 did better on the text they had practised retrieving. Ninety of them had predicted concept mapping would be as good or better.

Since then the finding has been pooled. Adesope, Trevisan and Sundararajan's 2017 meta-analysis in the Review of Educational Research found practice tests beat restudying and every other comparison condition. Yang, Luo, Vadillo, Yu and Shanks pooled 222 studies covering 48,478 students in real classrooms rather than laboratories in Psychological Bulletin in 2021, and found a medium effect of g = 0.50 on academic performance. And when Dunlosky and four colleagues rated ten common study techniques for the Association for Psychological Science in 2013, two came out with high utility: practice testing and spacing study out over time. Rereading and highlighting — the two techniques students report using most — rated low.

Why it works, and why it feels like it doesn't

The explanation is not complicated. Reading puts information in front of you. Recalling it makes you rebuild it from whatever is already in your head, and the act of rebuilding changes the memory — it strengthens the route to it and connects it to the cues you used. Robert Bjork's phrase for this is a desirable difficulty: the effort is not a cost of learning, it is the learning.

Rereading produces something else, which is fluency. The second time through a page, the words are familiar, the sentences are easy, and familiarity feels exactly like knowing. It is not. Recognising a sentence when you see it and producing it when you cannot see it are different acts, and only the second is what an exam asks for. This is why the rereaders in the 2006 experiment were the more confident group, and it is why the habit survives contact with results. When Karpicke, Andrew Butler and Roediger surveyed 177 undergraduates in 2009, 84% listed rereading among their strategies, 55% called it their main one, and 11% mentioned testing themselves. Of those who did, most said they used it to check what they knew, not to learn — which, as the Swahili experiment showed, means they stopped at exactly the point where the benefit begins. We wrote about the feeling itself in the first part of The Confidence Trap.

Two more findings settle the objections people raise next.

Doesn't this just teach you the answers to those questions? No. Butler's 2010 study in the Journal of Experimental Psychology: Learning, Memory and Cognition had students either restudy passages or answer questions on them, then a week later gave everyone new inferential questions — including questions from a different subject that could only be answered by applying a concept from the passage. The tested students beat the restudiers on those too. What retrieval strengthens is the concept, not the sentence.

Doesn't testing yourself make you more anxious? The reverse. Smith, Floerke and Thomas showed in Science in 2016 that memories learned by retrieval practice held up under acute stress, while memories learned by restudying did what stressed memories usually do and fell apart. And in a survey of 1,408 students whose teachers had run retrieval practice in class, Agarwal and colleagues found 72% said it made them less nervous about tests, not more. Nothing is as calming in an exam hall as having already produced the answer, from memory, several times before.

"But I need the basics first" — yes, and less than you think

This is the objection worth taking seriously, because it is half right. You cannot retrieve what was never put in. A student who opens a bank of questions on a topic they have never met will guess, and guessing is not retrieval. So yes: the lecture, one read of the handout section, the line of the course outline it sits under. That is the basics.

What the evidence does not support is the second read, the third, or the belief that you have to feel ready before questions are allowed. In 2009 Kornell, Hays and Bjork ran a series of experiments in JEP: Learning, Memory and Cognition built so that participants would fail: they were asked questions they could not possibly know, guessed wrong, and were then shown the answer. Those people remembered the answers better than people who were simply shown the question and the answer together. Trying and failing, followed by the correct answer, beat reading. The bar for "ready for questions" is one pass, not three.

So the rule is: read it once, then try a question on it. If the question shows you that the pass was not enough, read that part again — you now know which part. Do not read the whole thing a third time before you have tested any of it. You will be reading to feel better, and the 2006 experiment is what feeling better costs.

The method: seven rules

Everything above reduces to seven habits. None of them takes longer than what you do now; several take less.

1. Read once, then shut the notes

Work in the units your course outline is written in — one objective, one topic line — not in chapters. Read the material for that objective once, properly. Then close it. Every question from here on is answered with the notes shut, because a question answered with the notes open is a reading exercise with a box to tick at the end.

2. Commit before you look

Write your answer down before you check it. The letter, the number, the two lines of working — on paper or in the box. Then look. The reason is that an answer you have not committed to cannot be wrong, and an answer that cannot be wrong teaches you nothing about what you know. Answer each question once. If you can click again until it goes green, it is not a question, it is a slot machine, and your memory treats it as one. This is why the first attempt in our objective drills accepts one answer, and why part three of The Confidence Trap argues a confident wrong answer, corrected, is worth more than a lucky right one.

3. Read the explanation, not just the key

The mark scheme tells you which; you need why. In 2013 Butler, Godbole and Marsh compared feedback that gave the correct answer with feedback that gave the correct answer and an explanation, in the Journal of Educational Psychology. Two days later, on the same questions repeated, the two were equal. On new inference questions testing the same concepts, the explanation group did better. The key gets you the mark on that question; the explanation gets you the marks on the questions you have not seen yet.

For multiple choice, go one step further and work out why each wrong option is wrong. Little, Bjork, Bjork and Angello showed in Psychological Science in 2012 that when the distractors are plausible, ruling them out is itself an act of retrieval — students who had done it later answered questions about the distractors better than students who had only been asked to recall the right answer. There is a risk on the other side: a plausible wrong option you looked at and never resolved can lodge as a fact. Butler and Roediger found in Memory & Cognition in 2008 that feedback, immediate or delayed, removed almost all of that damage. So: never leave a multiple-choice question without knowing why the other three were not the answer. On a PremioQuest course paper this is what the examiner's rationale under each question is for, and "Unpack this question" is the longer version for the ones you got wrong.

4. Keep the ones you got right

The instinct after a correct answer is to drop it and move on to what you do not know. The Swahili experiment is the whole argument against that instinct: pairs dropped from testing after one correct recall fell to a third by the following week, and pairs kept in the tests held at four-fifths. Getting it right once is where the learning starts, not where it ends. Keep a right answer in the rotation until you have retrieved it correctly on at least two separate days. Keep a wrong one until you have retrieved it correctly twice after the day you got it wrong.

5. Space it

When you revisit matters as much as whether you do. Cepeda, Vul, Rohrer, Wixted and Pashler taught 1,350 people a set of facts, brought them back after gaps of up to three and a half months, and tested them up to a year later, in Psychological Science in 2008. The best gap between the first session and the return visit was a large fraction of the time remaining before the test: around a fifth to two-fifths of it for a test a week away, falling to a twentieth to a tenth for a test a year away. If your paper is four weeks away, the second visit to an objective belongs about a week after the first, not the same evening — and not the night before. The forgetting in between is not a failure of the method. It is what makes the second retrieval work.

6. Mix the topics

When you practise one topic in a block, the topic tells you which method to use before you have read the question. An exam does not. Rohrer, Dedrick and Stershic gave 126 seventh-grade students the same maths problems either blocked by type or interleaved, in the Journal of Educational Psychology in 2015, and the interleaved students did substantially better on an unannounced test a month later (d = 0.79) — because they had practised the step the blocked students never had to: deciding what kind of problem this is. Once you have two or three objectives under way, mix your questions across them. It will feel slower and less tidy. That is the difficulty doing its work.

7. Sit a full paper under the clock, then sort the misses

Before the real paper, sit one whole paper — the full length, the full mix, timed, with nothing open. Then do the thing most students never do with a practice paper: go through every lost mark and sort it. Did you not know it, did you misread the question, or did you run out of time? Those are three different problems with three different fixes, and part four of The Confidence Trap is a method for the sorting. On a PremioQuest course, "Set yourself a timed paper" draws across the whole outline, and the missed-questions row keeps everything you got wrong until you get it right.

A week of this, for one course

Here is what the seven rules look like laid across a week for a single course with an exam a month out. The reading column is one pass; the questions column is always with the notes shut.

Day	Read once	Questions (notes shut)	Return visit
Mon	Objectives 1–2	10 on objectives 1–2	—
Tue	Objectives 3–4	10 on objectives 3–4	—
Wed	Objective 5	10 on objective 5	5 on objectives 1–2
Thu	Objectives 6–7	10 on objectives 6–7	5 on objectives 3–4
Fri	Objective 8	10 on objective 8	5 on objective 5; every question missed so far
Sat	—	A timed paper across objectives 1–8	—
Sun	Only the sections the paper exposed	—	Every miss from the paper, from memory, with the explanations

Notice what is not in the table. There is no second read of anything on the day you first read it. There is no day on which you only read. The return visits are short — five questions — and land two to three days after the first pass: closer together than the spacing evidence would put them for a paper a month away, because a one-week table cannot wait a week. From the second week on, let the gaps stretch to a week or more. Saturday's paper is mixed on purpose. Sunday reads only what the paper showed was not there, which is the one kind of rereading the evidence supports.

Scale it to your course. The ratios are the point: one pass in, everything else out; return visits short and a few days late; a mixed paper before the real one.

Past questions are not the same thing

Back to the pasco. A past paper is a bank of questions, and everything above applies to it — if you use it as one. Answered cold, once, with the reason for each answer looked up afterwards and the misses revisited in three days, a past paper is retrieval practice, and a good one, because the questions were written by the people who write your questions.

Worked through with the marking scheme open, on the last night, to fix the shape of the answers in your head, it is rereading with extra steps. The paper is identical. The difference is entirely in whether you produced the answer or looked at it.

Two honest limits, because the past-paper trap has more than one jaw. A past paper gives you the key and not the why: the marking scheme says B and nothing about why C was tempting, which is exactly the part rule three says to chase. And a question set written by a model from your own notes — which is what our notes door produces, and what you get when you ask any assistant to quiz you — leans towards recall, because recall questions are the easy ones to generate. They are better than rereading, by every study above. They are not the same as questions an expert wrote against the outline, with a rationale under each one. If you have both, use the model's questions early, for the first pass, and the expert's for everything after.

Start tonight

Pick one course and one objective from its outline. Read the material for it once. Shut the notes. Answer ten questions on it, writing each answer down before you check. For every miss, read why, and for every multiple choice, work out why the other options were wrong. Write the number you got right on the outline next to the objective. Come back to it on Thursday.

That is the method. It is slower per question than what you do now and faster per mark than anything else in the literature.

Sources

1. Roediger, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249–255. journals.sagepub.com
2. Karpicke, J. D., & Roediger, H. L. (2008). The critical importance of retrieval for learning. Science, 319(5865), 966–968. science.org
3. Karpicke, J. D., & Blunt, J. R. (2011). Retrieval practice produces more learning than elaborative studying with concept mapping. Science, 331(6018), 772–775. science.org
4. Adesope, O. O., Trevisan, D. A., & Sundararajan, N. (2017). Rethinking the use of tests: A meta-analysis of practice testing. Review of Educational Research, 87(3), 659–701. journals.sagepub.com
5. Yang, C., Luo, L., Vadillo, M. A., Yu, R., & Shanks, D. R. (2021). Testing (quizzing) boosts classroom learning: A systematic and meta-analytic review. Psychological Bulletin, 147(4), 399–435. pubmed.ncbi.nlm.nih.gov
6. Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students' learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4–58. journals.sagepub.com
7. Bjork, E. L., & Bjork, R. A. (2011). Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning. In Psychology and the Real World (pp. 56–64). Worth. bjorklab.psych.ucla.edu
8. Karpicke, J. D., Butler, A. C., & Roediger, H. L. (2009). Metacognitive strategies in student learning: Do students practise retrieval when they study on their own? Memory, 17(4), 471–479. tandfonline.com
9. Butler, A. C. (2010). Repeated testing produces superior transfer of learning relative to repeated studying. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36(5), 1118–1133. pubmed.ncbi.nlm.nih.gov
10. Smith, A. M., Floerke, V. A., & Thomas, A. K. (2016). Retrieval practice protects memory against acute stress. Science, 354(6315), 1046–1048. science.org
11. Agarwal, P. K., D'Antonio, L., Roediger, H. L., McDermott, K. B., & McDaniel, M. A. (2014). Classroom-based programs of retrieval practice reduce middle school and high school students' test anxiety. Journal of Applied Research in Memory and Cognition, 3(3), 131–139. pdf.retrievalpractice.org
12. Kornell, N., Hays, M. J., & Bjork, R. A. (2009). Unsuccessful retrieval attempts enhance subsequent learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(4), 989–998. web.williams.edu
13. Butler, A. C., Godbole, N., & Marsh, E. J. (2013). Explanation feedback is better than correct answer feedback for promoting transfer of learning. Journal of Educational Psychology, 105(2), 290–298. doi.org
14. Little, J. L., Bjork, E. L., Bjork, R. A., & Angello, G. (2012). Multiple-choice tests exonerated, at least of some charges: Fostering test-induced learning and avoiding test-induced forgetting. Psychological Science, 23(11), 1337–1344. doi.org
15. Butler, A. C., & Roediger, H. L. (2008). Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing. Memory & Cognition, 36(3), 604–616. link.springer.com
16. Cepeda, N. J., Vul, E., Rohrer, D., Wixted, J. T., & Pashler, H. (2008). Spacing effects in learning: A temporal ridgeline of optimal retention. Psychological Science, 19(11), 1095–1102. journals.sagepub.com
17. Rohrer, D., Dedrick, R. F., & Stershic, S. (2015). Interleaved practice improves mathematics learning. Journal of Educational Psychology, 107(3), 900–908. files.eric.ed.gov
18. Amoako, J. K., Sharma, Y. K., & Danquah, P. (2020). "Memorize, Reproduce, and Forget" inclination; students' perspectives: A study of selected universities in Ghana. In Computational Methods and Data Engineering: Proceedings of ICMDE 2020, Volume 2. Springer. link.springer.com