Feedback: Asking the Right Questions

Previously, I wrote about what feedback to listen to, and what feedback actually tells you. But the issue I left for last is how to ask for feedback. While this might seem out of order, it is important to cover the other two topics first because they inform what we look for and ask of our players. When you try to listen to all feedback, you need to know how to get as much of it as possible. And you need to make sure you get the right kind of information out of your players so you can separate the good design advice from the opinions. Trying to accomplish both of these things is not a simple matter.

The most direct way of getting feedback is to ask your players what was good and bad about the game. But when you ask for both positive and negative feedback, don’t assume you know which is which. Sometimes positive feedback can be negative if what players did doesn’t match with your goal, and negative feedback can be positive, if it makes the decisions more interesting. In testing New Bedford, I’ve heard from several people who didn’t like having to fight for position on the whaling track. Yes, that’s negative feedback, but it’s an element that is important to the game, so it shows that the game is working as intended.

The hardest part of asking for feedback is figuring out what questions to ask, because what you ask for and how you ask for it are two different issues that both impact the results. That’s a reason that political polls can be misleading, and it also has far-reaching implications when businesses try to understand the customer experience. Part of my wife’s job is figuring out the right questions to ask to get quantifiable feedback. And, after all, isn’t feedback just a way of measuring the player’s experience of the game?

One of the most common pieces of advice I’ve gotten is to ask open-ended questions to encourage people to tell you more. You might get more direct responses from yes/no questions, but it won’t give you the context to understand why people give a certain answer, and you might miss something that you didn’t explicitly ask for. Remember that you don’t know what kind of message the player is trying to send you, so while it is good to include focused questions (“What buildings did you like?”) it is also helpful to ask more general questions (“What parts of the game did you like?”).

Often, when playtesting, the designer is interested in fixing problems in the game to improve it. A direct approach is to ask players where the problems were in the game. It might seem more useful to ask for the problems, instead of the solutions, but Kevin G. Nunn examined this in a three part series [Part 1, Part 2,]. Another article over on 3D Total Games comes to the same conclusion, that problems are better than solutions, but getting both is even better. Without trying to repeat what those articles cover (which are all worth reading), one takeaway is that players don’t always know what they are trying to say, so limiting what they can say misses out on information.

Gather both logical feedback (“what worked and what didn’t?”) and the emotional feedback (“how did you feel while playing?”). Logical feedback is great for making adjustments to the game mechanics, but emotional feedback helps in developing the whole experience. A lot of the fun of a game can be remembering the story of it later, so probing the emotional connection the player had is a more useful tool than simply asking if the game was fun.

Don’t ask questions just about the game they played. If you are lucky enough to have repeat players, ask about what changed, what improved, and what got worse. But possibly even more importantly, ask about the next game they haven’t played yet. One of my favorite questions is asking players what they would do differently, because it makes them consider things they might not have actually done in the game. It can also give a measure of how replayable the game is, by showing whether there are different ways of approaching the game. Or it can show that the player is still thinking about the game even at the end, which is a great way to make a connection to the player. In marketing terms, I guess this would be a measure of player retention potential.

If you are testing at a Protospiel or Unpub event, most players will be playing it for the first time. This is a great opportunity to ask about approachability. This isn’t a topic I have covered, but Alex Harkey put together a very indepth discussion of what makes a game approachable. I’ll just say that focusing on ease of learning and appeal of the theme are important first-impressions to understand. With regards to theme, ask not only what people think of it, but also how it works with the mechanics.

Besides talking to players after the game, I have collected a lot of feedback via the Unpub feedback form. Though it has gone through several changes, there are several constants across versions. The current version includes 7 numerical ratings, that I’ve discussed from a player perspective. One advantage of using more numerical questions is that players are more likely to respond with a number than they are to write a long response. But the Unpub form also includes a few open-ended questions, notably what would the player change, and what (if anything) made the end-game predictable. And the Unpub form includes one of the most important questions for anyone looking to eventually sell their games, “Would you buy this?”

So this wraps up my thoughts on feedback. In some ways, I feel like I’ve only really scratched the surface. The questions I’ve included today should be jumping-off points to more thoughtful consideration of your game. I’m certainly still learning how to get feedback and what to do with it once I have it. Everyone has something useful to tell you, but only some of it will make your game better. By carefully choosing the questions to ask, you can identify the red threads and red flags, and separate the real game design from the player’s desires.

I’ll end with a quote from game designer Kevin Wilson (via Cardboard Edison).

If you ignore all feedback and advice, you will fail. If you twist and contort your designs to follow all feedback, you will fail. Listen to your advisors, and weigh their words of wisdom. But you have to make the final decisions, you have to provide the vision.

How did I do with this series? What was most useful? What did I get wrong? Be sure to leave me some feedback.

Advertisements

, , , ,

  1. Leave a comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: