Final Step in Learning Game Design: Playtest, playtest, playtest

Previous posts in this learning game design series have focused on sexy stuff such as game goals and core dynamics, game mechanics, game elements, or scoring and rewards. This final post is about the critical importance of playtesting your game as you move through development.

Designing  a game is different than designing an eLearning solution. There is a totally new term that comes into the process: playtesting. Playtesting is NOT usability testing, focus group testing, quality assurance testing, or internal design review. Playtesting is what you do to evaluate whether your game is really playable and that it  functions the way you intended for it to function – as a game and as a learning solution.

Playtesting helps you answer these questions: Is it fun? Is it balanced (e.g. not too hard and not too easy)? Is it complete? Did people learn what you intended for them to learn? Playtesting is not something you do once or twice. You do it several times, each time further refining your game play experience and the learning experience. For mega-games like Halo or The Sims, designers may have done up to 3,000 hours of playtesting to verify that their game worked. For a learning game you craft yourself – or with a small team – you should assume at least 30-40 hours of play testing. Gulp – that’s right. 30-40 hours of testing time.

Phases of Playtesting

Your first playtest is with the very first version of the learning game you create, which should be a paper prototype. The image above is a paper prototype participants created in the Play to Learn workshop that Karl Kapp and I do together.  It’s rudimentary, but it gets the job done. The game designers and learning designers very quickly discover what works and where the holes are – and there will be holes. They’ll also come up with new ideas or ways to tackle problems. Here’s an overview of the three major phases of playtesting you should plan to go through. You may do multiple rounds of playtesting within each phase.

Phase 1 –  Self-test. You and your design team play the initial prototype and evaluate it. It’s okay to do a lot of discussing while you’re playing – and modifying rules and ideas on the fly as you go. You should keep game materials very basic for this test. Paper is best. If the paper test goes well, you can shift to online formats. If it doesn’t, re-do the game on paper and playtest it again as a paper version before going online.

Phase 2 –  Play test with friends and colleagues. Once you go through initial playtests with your design team and refine your game a few times, you’re ready to pull in some outside perspectives. Ideally you will include someone from your target audience.  Your team’s job  is to sit back and observes (quietly!) while others come in and play the game. At this point, you want there to be some legitimate game assets – artwork, programmed interactions, real content, scoring, and rules to follow.  You’ll debrief the experience and then decide what changes to make.

Phase 3 –  Play test with (gulp) strangers. Ideally all these strangers represent your target audience. These folks will be 100% objective, which friends and colleagues are not. I’ve learned the hard way (by going too far with internal playtesting and getting “great” results) that you need to loop in people who do not care about your feelings or the amount of time you’ve spent on the game. However, they SHOULD reflect people who really need to learn the stuff in your game. Otherwise, they can rate it lower simply because it’s not of interest to them.

6 Tips For a Good Learning Game Play Test

1. Don’t share the background of the game before people play. That’s part of the playtest. Can your players “get it” without you explaining what the game is about?
2. Do tell them what to expect: 15-20 minutes of game play followed by Q&A.
3. Emphasize the need for playtesters to “think out loud” as they play. You want to hear their internal thoughts spoken aloud. Things such as “This is really confusing.” “I don’t understand the rules.” “I wonder what would happen if I make this choice? ” are all good things to say aloud.
4. Keep your own mouth closed as much as you can. Do help players if they get truly stuck, but try to limit your interactions with players during the game.
5. Stop play after about 20 minutes and conclude with debrief questions. Take copious notes.
6. Keep a playtesting journal or log that documents the results of each playtest you do and chronicles the decisions you make about game changes.

Post-play debrief questions

We used to have a pretty big list of questions we asked. We’ve distilled the list down to five:

1. What did you learn? Compare responses to what people were supposed to learn. Rationale: if people didn’t learn, the game doesn’t work – no matter how fun people think it was. It’s critical you get people to tell you what they learned in their own words so you can compare it against the learning goal and learning objectives of the game.
2. On a scale of 1 – 5, with 1 being low and 5 being high, what was your engagement level in the game play experience? Rationale: if people weren’t engaged, then they aren’t having fun – and games should be fun. They should intrigue or interest the player. Otherwise, the player will mentally check out pretty quickly and not learn much of anything.
3. Did your engagement level change at any point during play (going up, down)? Rationale: There could be a confusing game element, a rule that needs to be changed or enhanced, or some other game element that requires adjustment to maximize the experience. Conversely, the game could start slow and really build for players. You want to assess all of this and determine if and how the game goal, the core dynamic, the game mechanics, or the game elements need to be tweaked. Careful listening can help you decide if you need to adjust a game element, a rule, or even the game’s goal or core dynamic.
4. If it did change, why did it change?
5. What, if anything did you find confusing or hard to understand as you played? Get people to explain their responses. Don’t accept a “Yes. The rules were confusing.” Rationale: even on a good game, there can be some confusion about how to play. Your job is to figure out whether the player’s confusion warrants action on your part. Did their confusion affect their learning or the engagement factor? If not, you may decide to do nothing. Was confusion limited to a single player or did many players report the same confusion?