10 Study Methods considered: A summary and reflection on "Improving Students' Learning With Effective Learning Techniques, Promising Directions From Cognitive and Educational Psychology"

I recently uncovered an interesting monograph entitled, "Improving Students’ Learning With Effective Learning Techniques, Promising Directions From Cognitive and Educational Psychology," by John Dunlosky, Katherine A. Rawson, Elizabeth J. Marsh, Mitchell J. Nathan, and Daniel T. Willingham. In it they assess 10 study methods, rating their utility either "low," "moderate," or "high." It's an interesting piece, first summarized by Annie Murphy Paul of Time.com last week. What follows is my own attempt at summary, which may or may not prove useful. If you have the time and inclination, I encourage you to give the work a read, particularly since the devil really is in the details with this stuff. So without further pause, let's jump in. 

1. Elaborative Interrogation

Elaborative interrogation involves answering the question "why?" about a topic. Says the paper, "The key to elaborative interrogation involves prompting learners to generate an explanation for an explicitly stated fact."

On how it works, the paper states, "The prevailing theoretical account of elaborative-interrogation effects is that elaborative interrogation enhances learning by supporting the integration of new information with existing prior knowledge...Although the integration of new facts with prior knowledge may facilitate the organization (Hunt, 2006) of that information, organization alone is not sufficient—students must also be able to discriminate among related facts to be accurate when identifying or using the learned information (Hunt, 2006)."

Be that as it may, it's unclear how wide an application elaborative interrogation can have, which according to the paper, is limited, "to discrete factual statements." Also it would appear that background learning affects the efficacy of elaborative interrofation, in that "Both correlational and experimental evidence suggest that prior knowledge is an important moderator of elaborative-interrogation effects, such that effects generally increase as prior knowledge increases."

Thus, "...the overall conclusion that emerges from the literature is that high-knowledge learners will generally be best equipped to profit from the elaborative-interrogation technique. The benefit for lower-knowledge learners is less certain."

The paper gives elaborative interrogation a rating of moderate utility. "Elaborative-interrogation effects have been shown across a relatively broad range of factual topics, although some concerns remain about the applicability of elaborative interrogation to material that is lengthier or more complex than fact lists. Concerning learner characteristics, effects of elaborative interrogation have been consistently documented for learners at least as young as upper elementary age, but some evidence suggests that the benefits of elaborative interrogation may be limited for learners with low levels of domain knowledge. Concerning criterion tasks, elaborative-interrogation effects have been firmly established on measures of associative memory administered after short delays, but firm conclusions about the extent to which elaborative interrogation benefits comprehension or the extent to which elaborative-interrogation effects persist across longer delays await further research."

2. Self Explanation

Says the paper, "...the core component of self-explanation involves having students explain some aspect of their processing during learning. Consistent with basic theoretical assumptions about the related technique of elaborative interrogation, self-explanation may enhance learning by supporting the integration of new information with existing prior knowledge." 

With respect to application, "Self-explanation has also been shown to facilitate the solving of various kinds of math problems, including simple addition problems for kindergartners, mathematical-equivalence problems for elementary-age students, and algebraic formulas and geometric theorems for older learners...broadly applicable"

It is unclear how long the benefits of knowledge obtained through self explanation lasts. Further, it has been suggested that some students benefit from instruction on how to develop useful explanations, particularly those without a natural knack at it. 

The paper rates self explanation as having moderate utility, in that it is widely applicable across a wide age range, but it remains unclear how durable such gains are. Furthermore, students with a poor natural capacity for developing explanations may require additional help to make the technique work.

3. Summarization    

According to the paper, "Successful summaries identify the main points of a text and capture the gist of it while excluding unimportant or repetitive material (A. L. Brown, Campione, & Day, 1981)."

Comparing summarization and note-taking to mere copying of quotations, it was found the former group performed better, for while, "Students in the verbatim-copying group still had to locate the most important information in the text...they did not synthesize it into a summary or rephrase it in their notes. Thus, writing about the important points in one’s own words produced a benefit over and above that of selecting important information." 

It is difficult, however, to assess the actual efficacy of summarization, because, "'summarization is not one strategy but a family of strategies” (Pressley, Johnson, Symons, McGoldrick, & Kurita, 1989, p. 5).'" "Depending on the particular instructions given, students’ summaries might consist of single words, sentences, or longer paragraphs; be limited in length or not; capture an entire text or only a portion of it; be written or spoken aloud; or be produced from memory or with the text present." So summarization appears to work, but it's not entirely clear how it can be optimally done. 

Consider, the study by Bednall and Kehoe (2011, Experiment 2) found that summarization alone did not yield better performance on tests. Rather, it found a correlation between the quality of the summary (getting the key facts correct, and linking those facts to prior knowledge) and later performance. So a summary can be effective if it actually captures the essence of the matter at hand. 

In addition, it's unclear whether short, simple summaries are better than more detailed summaries that capture more of the actual text. It also appears that prior knowledge, writing skills, and instruction in making better summaries improves the effects of summarization.

In sum, the paper deemed summarization as a low utility technique, primarily because it relies on an accurate assessment of important details, a fact which tends to get better with age and further training. As a result, younger students and those not adept at summarizing have a hard time benefiting from the technique, and might be better served by the use of another.

4. Highlighting and Underlining

Highlighting and underlining are among the most common study techniques used by students for improving learning, yet it turns out they are among the least effective methods to such ends. They "typically appeal to students because they are simple to use, do not entail training, and do not require students to invest much time beyond what is already required for reading the material." Interestingly, "Students are more likely to remember things that the experimenter highlighted or underlined in the text." It is believed this due to the "isolation effect," which holds that when presented with a list of related objects (pen, pencil, quill, etc.), inserting an unrelated object (frog) increases the likelihood that the student will remember it. "The analogy to highlighting," says the paper, "is that a highlighted, underlined, or capitalized sentence will 'pop out' of the text in the same way" as the word "frog" did in the list of writing implements. 

Furthermore, "Marking too much text is likely to have multiple consequences. First, overmarking reduces the degree to which marked text is distinguished from other text, and people are less likely to remember marked text if it is not distinctive (Lorch, Lorch, & Klusewitz, 1995). Second, it likely takes less processing to mark a lot of text than to single out the most important details." Limiting the amount one can highlight (for example, one sentence per paragraph), did show some benefit.

Highlighting was deemed by the paper to be of low utility, because it, "...does little to boost performance," and "...may actually hurt performance on higher-level tasks that require inference making." More effective highlighting of key points may improve the technique's utility, but that remains unclear.
5.Keyword mnemonic 

The Keyword mnemonic technique involves the construction of interactive mental imagery in such a way that the image triggers a memory. For example, as a way of remembering that la dent in French means "tooth" in English, one might imagine a dentist holding a tooth. The similarity between dent and "dentist" can facilitate the memory that la dent in English means "tooth." In the Raugh and Atkinson (1975) study, which involved learning Spanish vocabulary terms, it was found the construction of such images improved learning over simply studying the Spanish and their English translation. 

The paper goes on to say, "The benefits of the keyword mnemonic generalize to many different kinds of material: (a) foreign-language vocabulary from a variety of languages (French, German, Italian, Latin, Russian, Spanish, and Tagalog); (b) the definitions of obscure English vocabulary words and science terms; (c) state-capital associations (e.g., Lincoln is the capital of Nebraska); (d) medical terminology; (e) people’s names and accomplishments or occupations; and (f) minerals and their attributes (e.g., the mineral wolframite is soft, dark in color, and used in the home). Equally impressive, the keyword mnemonic has also been shown to benefit learners of different ages (from second graders to college students) and students with learning disabilities (for a review, see Jitendra, Edwards, Sacks, & Jacobson, 2004)."

However, "Proponents of the keyword mnemonic do acknowledge that its benefits may be limited to keyword-friendly materials (e.g., concrete nouns), and in fact, the vast majority of the research on the keyword mnemonic has involved materials that afforded its use." Furthermore, "Few studies have directly examined whether students can successfully generate their own keywords, and those that have have offered mixed results."

The paper rates the technique as having low utility. "It does show promise for keyword-friendly materials, but it is not highly efficient (in terms of time needed for training and keyword generation), and it may not produce durable learning."

6. Imagery use for text learning

Imagery use involves the conversion of text into mental images. As measured in some studies, there are,"...significant benefits of imagery use on measures involving the recall or summarization of text information (e.g., Kulhavy & Swenson, 1975), but... educed or nonexistent benefits on comprehension tests and on criterion tests that require application of the knowledge (Gagne & Memory, 1978; Miccinati, 1982)." As a result, the paper gives imagery use a rating of low utility, stating, "...the benefits of imagery are largely constrained to imagery-friendly materials and to tests of memory, and further demonstrations of the effectiveness of the technique (across different criterion tests and educationally relevant retention intervals) are needed.

7. Rereading 

Like highlighting, rereading is one of the most reported study techniques employed. As its name implies, it involves learning by reading the material under study multiple times. According to the paper, several studies--Bromage & Mayer, 1986; Kiewra, Mayer, Christensen, Kim, & Risch, 1991; Rawson & Kintsch, 2005)--suggest rereading shows, "...greater improvement in the recall of main ideas than in the recall of details." Furthermore, "Spaced rereading" (involving an interval of time between the first reading and the next) appears to have a greater benefit than "Massed reading," which involves no significant delay between readings.

The paper designates rereading as having a "low" utility. "Although benefits from rereading have been shown across a relatively wide range of text materials, the generality of rereading effects across the other categories of variables in Table 2 has not been well established." 

"Concerning criterion tasks, the effects of rereading do appear to be durable across at least modest delays when rereading is spaced. However, most effects have been shown with recall-based memory measures, whereas the benefit for comprehension is less clear. Finally, although rereading is relatively economical with respect to time demands and training requirements when compared with some other learning techniques, rereading is also typically much less effective. The relative disadvantage of rereading to other techniques is the largest strike against rereading and is the factor that weighed most heavily in our decision to assign it a rating of low utility."

8. Practice Testing

For over a hundred years (Abbott 1909), it has been known that practice testing, "...enhances learning and retention." Practice testing can include, "...actual or virtual flashcards, completing practice problems or questions included at the end of textbook chapters, or completing practice tests included in the electronic supplemental materials that increasingly accompany textbooks." Why practice testing works tends to fall under two categories: direct effects ("changes in learning that arise from the act of taking a test itself"), and mediated effects ("...learning that arise from an influence of testing on the amount or kind of encoding that takes place after the test (e.g., during a subsequent restudy opportunity))." 

Both effects appear to alter the ways in which information is organized and retrieved. Regarding direct effects for example, "Carpenter (2009) recently proposed that testing can enhance retention by triggering elaborative retrieval processes. Attempting to retrieve target information involves a search of long-term memory that activates related information, and this activated information may then be encoded along with the retrieved target, forming an elaborated trace that affords multiple pathways to facilitate later access to that information." Similar explanations are given by Pyc and Rawson (2010, 2012b) with respect to the mediated effects of practice testing. The evidence each case showed greater performance for practice testing than for mere studying.

Regarding the most efficient way to employ practice testing:

"Several studies have increased the number of tests presented in immediate succession within a session and have found minimal or nonexistent effects, in contrast to the sizable benefits observed when repeated tests are spaced (e.g.,Carpenter & DeLosh, 2005; Cull, 2000; Glover, 1989; Karpicke & Bauernschmidt, 2011). Concerning the time intervals involved with spacing, longer is better. Repeated practice testing produces greater benefits when lags between trials within a session are longer rather than shorter (e.g., Pashler, Zarow, & Triplett, 2003; Pavlik & Anderson, 2005; Pyc & Rawson, 2009, 2012b), when trials are completed in different practice sessions rather than all in the same session (e.g., Bahrick, 1979; Bahrick & Hall, 2005; Kornell, 2009; Rohrer, 2009; Rohrer & Taylor, 2006), and when intervals between practice sessions are longer rather than shorter (Bahrick et al., 1993;Carpenter, Pashler, & Cepeda, 2009, although the optimal lag between sessions may depend on retention interval—see Cepeda et al., 2009; Cepeda, Vul, Rohrer, Wixted, & Pashler, 2008)."

Furthermore, an important aspect of practice testing involves feedback, and substitution of incorrect answers for correct information. "Practice testing with feedback also consistently outperforms practice testing alone."

The paper gives practice testing a high utility: 

"Testing effects have been demonstrated across an impressive range of practice-test formats, kinds of material, learner ages, outcome measures, and retention intervals. Thus, practice testing has broad applicability. Practice testing is not particularly time intensive relative to other techniques, and it can be implemented with minimal training. Finally, several studies have provided evidence for the efficacy of practice testing in representative educational contexts."

9. Distributed Practice

Distributed Practice involves practice sessions at an interval. It is based on the Distributed-Practice Effect, which, "...refers to the finding that distributing learning over time (either within a single study session or across sessions) typically benefits long-term retention more than does massing learning opportunities back-to-back or in relatively close succession."

It's unclear why distributed practice works, but several theories offer explanations. They range from suggesting that unspaced study sessions lead students to believe they know material better than they actually do, while another holds that the second session simply reinforces the progress of the first session. Says the paper, "Given the relatively large magnitude of distributed-practice effects, it is plausible that multiple mechanisms may contribute to them; hence, particular theories often invoke different combinations of mechanisms to explain the effects."

Importantly, "Distributed practice refers to a particular schedule of learning episodes, as opposed to a particular kind of learning episode. That is, the distributed-practice effect refers to better learning when learning episodes are spread out in time than when they occur in close succession, but those learning episodes could involve restudying material, retrieving information from memory, or practicing skills." As such, distributed practice is not a study technique per say, but rather a method of organizing the resource of time. It doesn't offer explicit insight into what we should do when trying to learn, only that we'll do better if our sessions are spaced and not done all at once (as is common when "cramming" for exams). While an optimal interval length is suggested in some studies, a general rule of thumb appears to be that the longer one wishes to retain information, the longer should be the intervals between practice. 

The paper gives distributed practice a high utility rating. "It works across students of different ages, with a wide variety of materials, on the majority of standard laboratory measures, and over long delays. It is easy to implement (although it may require some training) and has been used successfully in a number of classroom studies."

10. Interleaved Practice

A less-investigated study method is called Interleaved practice, "...in which students alternate their practice of different kinds of items or problems." The paper uses the study of Rohrer and Taylor (2007) to describe how the method can be used in practice. The study involved teaching students how to find the volume of four solid figures: 

"Students had two practice sessions, which were separated by 1 week. During each practice session, students were given tutorials on how to find the volume for four different kinds of geometric solids and completed 16 practice problems (4 for each solid). After the completion of each practice problem, the correct solution was shown for 10 seconds. Students in a blocked-practice condition first read a tutorial on finding the volume of a given solid, which was immediately followed by the four practice problems for that kind of solid. Practice solving volumes for a given solid was then followed by the tutorial and practice problems for the next kind of solid, and so on. Students in an interleaved-practice group first read all four tutorials and then completed all the practice problems, with the constraint that every set of four consecutive problems included one problem for each of the four kinds of solids. One week after the second practice session, all students took a criterion test in which they solved two novel problems for each of the four kinds of solids. Students’ percentages of correct responses during the practice sessions and during the criterion test are presented in Figure 13, which illustrates a typical interleaving effect: During practice, performance was better with blocked practice than interleaved practice, but this advantage dramatically reversed on the criterion test, such that interleaved practice boosted accuracy by 43%."

Why this occurs is not definitively known, but according to the paper a "discriminative-contrast hypothesis" is likely at work. By interleaving the types of problems encountered at once, students did not attain the short-term procedural competency of solving specific problems (as with block practice). But they did learn how to discriminate between types of problems, which appears to have provided greater benefits in actual real-life situations (for example, on the criterion test). This explanation is strengthened by the study of Kang and Pashler (2012), in which college students studied the work of artists in either blocked sessions of a single artist's style, or interleaved sessions of studying multiple artists' styles (another group employed blocked sessions, but had a cartoon appear between paintings). The results: 

"Criterion performance was best after interleaved practice and was significantly better than after either standard or temporally spaced blocked practice. No differences occurred in performance between the two blocked-practice groups, which indicates that spacing alone will not consistently benefit concept formation."

As hinted at above, it is believed the interleaved method worked most effectively because it showed the contrasts between various artists' styles. As a result, when faced with a painting of a particular artist, a student from the interleaved practice would be more likely to discriminate between varying styles.

The paper gives interleaved practice a rating of moderate utility. "On the positive side, interleaved practice has been shown to have relatively dramatic effects on students’ learning and retention of mathematical skills, and teachers and students should consider adopting it in the appropriate contexts. Also, interleaving does help (and rarely hinders) other kinds of cognitive skills. On the negative side, the literature on interleaved practice is currently small, but it contains enough null effects to raise concern. Although the null effects may indicate that the technique does not consistently work well, they may instead reflect that we do not fully understand the mechanisms underlying the effects of interleaving and therefore do not always use it appropriately."

Conclusion:

I found it interesting how some study techniques--even ones commonly employed like rereading and highlighting (both methods I use!)--are not as effective as others, in some cases dramatically so. Another interesting point involves how some techniques help with certain types of questions or assessment, but not with others. Accordingly, it isn't just a matter of which technique is "better" or "worse" than other, but rather which is more effective with a certain end in mind. In general, practice testing at an interval on a variety of topics appears to yield the optimal result, but for those skilled at summary and elaborative interrogation, those methods may also have utility. In the end, I suspect each of us must find what techniques work best for us. It can be helpful though, to know what the science currently says on the matter.

Happy Tuesday, friends :)