Every year a large number of students sit standardised reading tests. The data from the tests are used to identify students requiring supplementary reading instruction and whether a school might need to change its reading program. However, how students engage with the test task is largely unknown and students with the same test score may have different underlying difficulties or strengths (e.g., decoding skills, working memory, vocabulary, prior knowledge, etc.).
Ardoin et al. (2019) sought to investigate reading comprehension by randomly assigning 166 grade 2-5 students into one of two conditions: Reading with questions presented concurrently with the text or with questions presented after reading the text with the text unavailable when answering the questions. The three texts chosen were of different lengths (56, 113 and 122 words) and varied in difficulty, with the longer texts begin associated with greater difficulty. Each text had 5 multi-choice questions. Six of the 15 questions were literal, where the answers were explicitly stated in the text. The remaining 9 questions were classified as inferential, however there was sufficient contextual information in the text to be able to answer the questions and none of these questions required synthesisation of the entire passage. The passages were presented on a LCD monitor in such a way that the student could read the passage and answer the associated questions without having to change the screen. As a part of the study they also tracked the students’ eye movement. In addition, students were assessed on a battery of reading achievement tests and working memory.
Results
Eye Movement
- Younger students had longer fixation times per word than older students, suggesting that reading the words required greater effort for younger students.
- Students generally spent less time on reading, made fewer regressions and had fewer fixations per word for the short passage compared to the longer, more complex passages.
- Readers spent more total time reading the words, made more regressions back to earlier words and had more fixations per word when the questions were presented at the same time as the passage and this was particularly evident for the longer passages.
Test-taking Behaviour
- When provided with the questions and the text at the same time, most students (96%) read the passage before reading the questions. 17% read the passage and answered the questions without referring back to the passage, 79% read the passage, read the questions and then returned to the text to answer the questions, 3% read the questions prior to reading passage, reread the questions and also returned to the text.
- Students who could refer back to the text spent 25% more time on reading the passage compare to those students who couldn’t refer back to the text.
- Accuracy in answering comprehension questions was higher for students when they could refer back to the text versus those students in the group where the text was removed.
- Student who were able to quickly and accurately locate and reread specific sections of the text related to a particular question, generally answered the questions more accurately.
- Working memory was significantly and positively correlated with accuracy scores when students did not reread the text, but there was no correlation when students did reread the text.
Discussion
- Rereading the text may not be particularly important for students who have generated a coherent mental representation of the text when reading.
- Students with poor working memory and/or are poor readers will benefit from learning how to quickly and accurately locate specific information within the text.
- If students are required to answer questions without being able to refer to the text, then they need to be taught to develop a stronger mental model of the text as they are reading.
- If students have a strong understanding of the text, recommending that students always refer back to the text may simply cost students test-taking time without any benefit in improved comprehension scores. This is particularly important on time-based tests.
Reference
Ardoin, S., Binder, K., Zawoyski, A., Nimoks, E., Foster, T. (2019). Measuring the behaviour of reading comprehension test takers: What do they do and should they do it? Reading Research Quarterly, 54(4), 507-529.