PLAYTESTING: Lessons from UX research

In this article I want to share some of the insights about testing prototypes that I have gained from my studies and experience in UX design, and how they can improve your game playtesting.

We’ll use this forum thread on BGG to define our game development phases, though feel free to relate to them in whatever way feels familiar to you.

Some of the recommendations and information about best practice in this article have come from great articles that can be found at Smashing Magazine and User Testing.com.

For this article I am going to chart the types of playtesting you might do on a graph. The goal here is to help you move your game along and get the most out of your playtesting.

Too often I have been guilty of, and observed other designers, continuing to be heavily involved in playtesting their games even at late stages of development, hampering their ability to continue to improve the design. With some practice, you can move your design into phases where you minimise potentially biased playtest data and get a more accurate picture of the player experience.

Principles of effective playtesting

First, some general tips and principles regarding playtesting that I have found helpful to keep in mind:

1. Any testing is better than no testing. Even if you think your game isn’t ready or complete, it is worth testing whatever you have and getting some reactions from players.

2. Look for behavioural insights rather than opinions. What players actually do is more valuable than what they think they did, or what they thought about the game.

3. Playtesting is about identifying ways to improve your game, while playing is for enjoying the game experience.

4. Less involvement leads to better accuracy. The further you remove yourself from the playtest, the more realistic picture you will get of the end user experience of players interacting with your game for the first time in the wild.

5. The sooner you can identify your target audience the better. When you have a clear picture of who your game will appeal to, you can begin to test with players in that demographic, and let their reactions and behaviour guide your game development decisions.

If you’re ever at a loss with how to progress your game design, try testing it and removing yourself from the test. 

Instead, sit to the side and observe how players behave, where they are engaged, what questions they ask, and what they think the answers should be. Avoid the temptation to run the game and provide all the answers.

Playtesting approach over time

Playtesting approaches

Each designer will have their own different approach to how they playtest their games. Broadly, the two types of playtesting will fall into the categories of Moderated or Unmoderated.

Moderated: The designer is present and involved in the test session. They could be playing the game as well, or introducing and facilitating the game. This is common for early designs or when recruiting enough playtesters is challenging.

Unmoderated: A type of testing where the designer is not directly involved. This may be blind playtesting, or when the designer is present but lets the players learn the game and play on their own. The designer does not influence how the test goes.

In my 6 years designing board games as a hobby, I have encountered designers using these methods during playtesting:

1. Designer playing with players (moderated)

2. Rules explained verbally, game observed, questions answered  (moderated)

3. Rules read by players, designer on hand to clarify as needed (partially moderated)

4. Rules explained verbally then game observed (partially moderated)

5. Supervised Blind playtest (unmoderated)

6. Remote Blind playtest (unmoderated)

Take a minute to reflect on what types of tests you usually do, and what are some of the strengths and weaknesses of that approach? How do you usually solicit feedback? How do you determine success or failure of the session?

One of the principles of UX design usability testing that applies here is summed up neatly in the article from Smashing Magazine:

The participant needs to know that your intention is to watch them using what you’ve built and, to ensure the conditions are as real as possible, you won’t be offering them advice.

This last point is critical, especially if you’re running a usability test on something you’ve designed. It’s important not to interrupt the test participant’s flow by offering them guidance and advice. You won’t be there to do this in the ‘real world,’ so resist the urge to offer advice from the sidelines.

When you think it is appropriate for your design, I strongly encourage you to move toward an unmoderated approach to playtesting to minimise your impact on the session and increase the validity of the data you are getting from your tests. 

The same article from Smashing Magazine offers this insight:

You’re dealing with humans, and if your test participants know that they’re giving you feedback on something you designed, they are likely — only naturally — to hold back on their criticisms. No one likes to hurt someone’s feelings, and it’s important you get an honest opinion, so don’t skew the participants’ answers by telling them they are judging something you designed.

For this reason, in late stages of development it is important to conduct tests without being present at all, so that you can get honest and objective feedback on your design.

Collecting feedback from unmoderated playtests

One of the advantages of conducting moderated playtesting is that the designer is present for the playtest and can listen to and solicit player feedback in person. The designer can ask questions about the player experience, what they liked and didn’t like, and will often find themselves inundated with suggestions from the playtesters. This all sounds great, right?

Well yes, and no. On multiple occasions I’ve heard incredible game designers such as Matt Leacock (Pandemic) and Eric Lang (Blood Rage) say they don’t even ask players for feedback at the end of the session. Instead, they get all they need from the test through observation of player behaviour, taking notes along the way. It is also telling that Matt Leacock used to be a UX designer himself, and his process involves taking video recordings of the playtests and watching the footage back, looking for player body language and engagement. This is advocacy for player behaviour over player opinion in terms of valuable playtest feedback.

So how can you collect player feedback if you’re not there to ask for it?

In addition to Leacock’s suggestion of having the player’s take a video recording of their session, you can use playtest feedback forms such as this one to collect player feedback. This allows the players to express themselves more freely without fear of hurting your feelings. You might consider creating your own form and making a Google Form version, issuing a survey link remotely to allow users to fill it out online.

The unmoderated playtesting approach is at odds with that of Bruno Faidutti (Citadels), about whom I have read is actively involved in all of his playtests because he wants to get a first hand feel of what playing the game feels like. 

All this to say that one approach is not obviously better than others, but rather to invite you to think about your own approach to playtesting and to engage with it intentionally. Use the time you have to get the most value out of your playtests, and if you are stuck, perhaps consider approaching your playtesting with a different method to see what new insights you gain into player behaviour.

Playtesting recommendations summary

Let’s end with some recommendations for how to get the most out of your playtesting:

1. Have an objective or plan for the testing session. What are you hoping to get out of it? What parts of the design are you still considering changes to and what changes are you testing? Create a script so you remain focused!

2. Consider removing yourself from playing so that you don’t influence the game experience. Use the opportunity to observe player behaviour and take notes about times players are particularly engaged or disengaged.

3. Record your sessions so that you have an objective record of the test and aren’t relying on your memory (or the memory of others) about how it went.

4. Test the prototype with 5 or more different players before making a change (depending on the stage the prototype is at).

5. Define the scope of your testing. How much time do you have? If you only had 15 mins, but a 3 hour game, how would you approach it to get the most out of that time?

6. Test your game with target players who most closely resemble your intended target audience.

7. Test your game with as many different players as you can. 

8. Design putting the player first. Keep your players at the front and center of your design thinking!

I’d love to hear about your playtesting experiences. What has worked for you? What methods do you use? What stage of your designs do you apply each of your methods?