nyt-featured

Spaced Repetition Featured in the New York Times

spacedlearning

You’ve seen us write articles and publish white papers on the science of remembering for well over a year now. We are big believers in the spacing effect, and using what we know about the brain to enhance what people remember. Doing so has a measurable impact on employee retention from training, among other things.

It seems the New York Times has taken interest in spaced repetition as well. In a weekend editorial titled “How Tests Make Us Smarter,” Dr. Henry Roediger discusses how testing can be used as a valid tool to promote learning. Specifically, testing requires learners to retrieve knowledge from memory. According to Roediger, knowledge is solidified in our working memory when we practice retrieving and using it.

According to NYTimes.com, Henry L. Roediger III is a professor of psychology at Washington University in St. Louis and a co-author of “Make It Stick: The Science of Successful Learning.”

Dr. Roediger goes on to mention how “many studies reveal that much of what we learn is quickly forgotten.” Therefore, “a central challenge to learning is finding a way to stem forgetting.” We could not agree more. Our research on the science of remembering (and forgetting) often comes back to the often-cited, and often misunderstood “Forgetting Curve.”

But testing alone does not provide the desired increase in retention. Here’s Roediger’s take. I’ve bolded some key points:

Retrieving knowledge from memory is more beneficial when practice sessions are spaced out so that some forgetting occurs before you try to retrieve again. The added effort required to recall the information makes learning stronger. It also helps when retrieval practice is mixed up — whether you’re practicing hitting different kinds of baseball pitches or solving different solid geometry problems in a random sequence, you are better able later to discriminate what kind of pitch or geometry problem you’re facing and find the correct solution.

While Dr. Roediger’s OpEd is specifically focused on testing students, the key points are applicable to corporate learning as well. In our Primer on Spaced Repetition and Feedback Loops, I connected the challenge of remembering  to the constant effort organization’s expend to deliver information employees need to know for their jobs. Rather than test scores, a great deal more is at stake. Organizations depend on their training to help employees have less accidents, make more sales, provide better customer service and make fewer mistakes.

Just like in the K-12 environment, the most common corporate learning methods are actually the least effective for learning when we take a look at the research. “Click next” eLearning courses, instructor-led sessions and PowerPoint slides are simply not enough to foster long term retention.

For this reason, we consistently integrate spaced repetition and feedback loops into both the custom learning solutions we develop and into the design of our Knowledge Guru game engine. We work with our customers to help them integrate Knowledge Guru games into a larger curriculum to maximize the retention benefits.

And while we carefully built spaced repetition into the design of Knowledge Guru, the research-based principles are available for anyone to take advantage of. As Roediger puts it, “As learners encounter increasingly complex ideas, a regimen of retrieval practice helps them to form more sophisticated mental structures that can be applied later in different circumstances.”

I encourage you to read the full article on NewYorkTimes.com. Then, have a look at Dr. Karl Kapp’s take on spaced repetition or “retrieval practice.”

onboarding-square

How to Make Employee On-boarding Memorable… In a Good Way

Onboarding

New employee on-boarding is hard on both the new hire and the manager. Most companies have a metaphorical onion of jargon, policies, procedures, and culture mores, and it can take weeks or even months for new employees to feel comfortable and be productive.

Large organizations have a dedicated HR department that updates the policies and procedures these employees need to know… and there are probably quite a few of them. Small organizations may not even have an HR department, leaving it to the new employee’s manager to get them everything they need quickly.

Either way, starting a new job often means rapidly acquiring huge amounts of new knowledge and skills. Some of this learning is directly related to job performance, while the rest is based on finding the right information, following the right processes and understanding the “company vernacular.”

Even companies with a well-developed “Corporate University” or fleshed out on-boarding program run into problems when new hires fail to remember what they learned in those introductory courses and sessions. Without proper reinforcement and spacing of key concepts, new employees will quickly forget the flood of new information. Instructor-led sessions and eLearning courses, even when well designed, fall short in driving true retention.

New research has shown that Ebbinghaus’ famous Forgetting Curve, which can approach 90% in lost information, is dependent on prior skills and experience. Learners with a prior framework for the new knowledge may only forget 30% without reinforcement, for example. Since new employees are usually new to the company, they are likely to fall closer to that 90% level of forgetting. When the policies and procedures actually matter, this becomes a huge problem for organizations to solve.

How We Fixed Our On-boarding Program

Our company, Bottom-Line Performance, has experienced nearly 100% growth in the last 2.5 years. We are still a small operation (just under 30 employees), but the growth has been rapid. While managers have traditionally handled the on-boarding and created 30 and 90-day programs for their new employees, our company has struggled to help new employees “learn the lingo” and understand basic processes and procedures. In the past, most folks have learned by piecing information together from various co-workers. They learn the ropes eventually, but the feedback is often “I wish I would have felt more comfortable learning and following the processes and terminology in the first three months.”

Our solution? Use our Knowledge Guru game engine to create a “BLP Guru” game to teach process, procedures and jargon! Employee on-boarding information is well-suited for the spaced repetition-based game engine.

Basics of using Knowledge Guru for on-boarding:

  • Our game covered policies and procedures, jargon and the “BLP Toolbox.” We wove workplace scenarios into all the topics.
  • New hires review the handbook, then play the game with handbook close by. The game reinforces concepts that may have been explained by a manager and also teaches new concepts.
  • The feedback provided for incorrect responses helps employees avoid embedding incorrect information.
  • Achievements, leaderboards and the “See the Standings” page give new hires a sense of connection and camaraderie (plus some needed motivation!)
  • The iterative questions and spaced repetition reinforce key learning objectives we identified as essential for all new hires to know or do.
  • When job titles, tools and policies change, we are easily able to update the game content (or change to a new game entirely) in the admin dashboard.

Scroll through the screenshots below to see some highlights of the BLP Guru Game. You can also request a demo for a complete walkthrough of how to use Knowledge Guru in an employee on-boarding program.

 

Basic policies and procedures work well within Knowledge Guru games.

Basic policies and procedures work well within Knowledge Guru games.

Images can be combined with relevant questions, such as this "find and locate" question.

Images can be combined with relevant questions, such as this “find and locate” question.

Players learn from their mistakes and receive feedback when they answer questions incorrectly.

Players learn from their mistakes and receive feedback when they answer questions incorrectly.

Rewards and achievements make otherwise dry content motivating.

Rewards and achievements make otherwise dry content motivating.

Players receive badges and topic mastery achievements when they complete on-boarding topics.

Players receive badges and topic mastery achievements when they complete on-boarding topics.

The "See the Standings" page shows where all new hires have progressed in the game. The leaderboards build camaraderie, too.

The “See the Standings” page shows where all new hires have progressed in the game. The leaderboards build camaraderie, too.

When the policies and procedures inevitably change, the game content is easy to update.

When the policies and procedures inevitably change, the game content is easy to update.

remembering-thumb

Four Strategies for Long-Term Retention in Corporate Learning (White Paper)

In some organizations, training is a “check the box,” “we’ve always done it this way” activity. That may work in some situations, but what about when remembering really matters? Corporate learning professionals need strategies to improve both knowledge transfer during the initial learning event as well as the rate of long-term retention.

Sharon Boller, Knowledge Guru creator and Chief Product Officer, has authored a new white paper on strategies you can use to help learners remember. It includes the learning and remembering strategies used in Knowledge Guru alongside other strategies we use time and again in our custom solutions.

Most importantly, you’ll find five unique business challenges in the white paper… and the strategies we used to solve those challenges for our clients.

The white paper is available as a free download in the Learning Center on our Bottom-Line Performance website. Click here or click the image below to download the white paper.

remembering-matters-clouds

This white paper, together with our Primer on Spaced Repetition and Feedback Loops, serve as the ideal introduction to the research-based learning principles we use in our Knowledge Guru games. We incorporate these strategies for learning and remembering into all of the custom learning solutions we develop as well. By studying the white paper, you will be able to:

  • Apply research-based strategies for both learning and remembering into your own learning designs.
  • Effectively use Knowledge Guru as a tool to increase learner retention.
  • Ensure any solution you implement at your organization is backed by research-based learning principles.

Download the white paper here.

Spaced Retrieval, Retrieval Practice, and Knowledge Guru: What Research Tells Us

Knowledge Guru’s game engine is designed to tap into two powerful and empirically supported instructional strategies—Retrieval Practice and Spaced Retrieval.

Retrieval Practice requires learners to recall information rather than simply re-read or re-listen to it. A review of the pertinent scientific literature reveals that the benefits of retrieval practice have been known for at least 100 years and they have been demonstrated with many diverse groups [1]. Retrieval Practice alone can provide improved recall performance by as much as 10-20%. [2] When combined with Spaced Retrieval the effect is multiplied. In fact, “the act of retrieving information from memory actually alters the retrieved memory by elaborating on the existing memory trace and/or creating additional retrieval routes. One consequence of these changes is that the probability of successful retrieval in the future is increased, making testing a potent mechanism for enhancing long term retention.”[3]

Spaced Repetition

Spaced Retrieval involves providing students with quiz or course content spaced over time and it, too, is among the most robust findings in educational psychology research. [4] It turns out that the greater the amount of spacing between retrieval events, the greater the potential benefit to retention. Spaced Retrieval helps learners retain access to memorized information over long periods of time because the spacing promotes deeper processing of the learned material. Ideally, the time between the learning events is greater than 24 hours, but shorter times have also been found to be effective. As long as 8 years after an original training, learners whose practices were spaced showed better retention than those who practiced in a more concentrated time period. [5]

Spaced Retrieval avoids two inherent problems with mass practice (learning all the information at once); the problems of learner fatigue and the likelihood of interference with preceding and succeeding learning.

One study combining Spaced Retrieval and Retrieval Practice indicated retention benefits of between 35-60% for students in the subject matter of Anatomy and Physiology over the control group. [6]

Additionally, using the combination of a quizzing game and online lectures, student’s minds will focus more on the course and wander less, which is always an issue in online learning. Researchers have found that by interspersing online lectures with short tests, student mind-wandering decreased by half, note-taking tripled, and overall retention of the material improved. As the researchers indicated, “In our experiments, when we asked students if they were mind-wandering, they said yes roughly 40 percent of the time. It’s a significant problem.” But using online quizzing in a manner such as Knowledge Guru will reduce mind wandering and increase performance.

In the study by Harvard researchers, participants were asked to learn from a 21-minute video lecture on statistics. The participants were told that the lecture would be divided in four parts, separated by a break. The students were all told that they might be quizzed at any time (when, in fact, only one condition had quizzing the first three times and all conditions had the quiz the fourth time.) [7]

During the break the participants were asked to perform math problems for a minute, and then would be assigned to one of three groups:

  • Perform more math problems for two more minutes (“untested group”)
  • Answer quiz questions for two minutes on the material they had just learned (“tested group”)
  • Review by seeing questions with the answers provided (“restudy group.”)

 

The result was that the students who were tested took more notes during the lecture and reported that their minds wandered less (19% as compared to 40%.)  In addition, the students who were tested on all segments out performed both of the other groups (tested = 90%, restudy = 76%, nontested = 68). Another finding was that students who were tested after each segment indicated that they were less anxious about the fourth test than students from the other two groups.

The researchers believe that the results are because the testing at the end of each segment, act as an incentive for students to pay closer attention to the lecture because they know they’ll have to answer questions at the end of each segment.

The researchers concluded: The present results… highlight the specific cognitive mechanism by which testing can facilitate learning. In particular, testing can be used to help students sustain attention to lecture content in a manner that discourages task-irrelevant (mind wandering) and encourages task-relevant (note taking) activities, and hence improves learning. Importantly, the benefits of testing for learning were accompanied by reductions in test anxiety (possibly because students became accustomed to testing style or as a result of positive feedback from earlier tests) and subjective estimates of cognitive demand.

References cited

[1] Larsen DP, Butler AC, Roediger HL 3rd. Repeated testing improves long-term retention relative to repeated study: a randomized  controlled trial. Med Educ  43: 1174–1181, 2009.

[2] Dobson, J. L. (2013) Retrieval practice is an efficient method of enhancing the retention of anatomy and physiology information Advances in Physiology Education 37: 184–191, 2013; doi:10.1152/advan.00174.2012.

[3] Roediger, H. L., & Butler, A.C . (2013). Retrieval practice (testing) effect. In H. L. Pashler (Ed.),Encyclopedia of the mind. Los Angeles, CA: Sage Publishing Co.

[4] Carpenter SK, DeLosh EL. (2005) Application of the testing and spacing effects to name learning. Applied Cognitive Psychology 19: 619–636, 2005. And Cull W. (2000) Untangling the benefits of multiple study opportunities and repeated testing for cued recall. Applied Cognitive Psychology 14: 215–235, 2000. And Cull W, Shaughnessy JJ, Zechmeister EB.(1996) Expanding understanding of the expanding-pattern-of-retrieval mnemonic:toward confidence in applicability. Journal of Experimental Psychology Applied 2: 365–378.

[5] Kapp, K. M. (2012) The Gamification of Learning and Instruction: Game-based Methods and Strategies for Training and Education. New York: Pfeiffer.

[6] Dobson, J. L. (2013) Retrieval practice is an efficient method of enhancing the retention of anatomy and physiology information Advances in Physiology Education 37: 184–191, 2013; doi:10.1152/advan.00174.2012.

[7] Szpunar, K. K., Khan, N. &, & Schacter, D. L. (2013). Interpolated memory tests reduce mind wandering and improve learning of online lectures. Proceedings of the National Academy of Sciences, published online April 1, 2013 doi:10.1073/pnas.122176411