Writing a playtest decision

Start with the choice that could change

A topic is not a decision. “The new onboarding” is a topic. “Do we keep the shared objective hidden until the first round, or teach it before matchmaking” is a decision. It identifies two plausible actions and gives the study somewhere to land.

Use this sentence: We need to decide whether to [action A] or [action B], for [population], before [real milestone]. If only one outcome is acceptable, the work may be validation rather than research. Say that openly so the session is not asked to provide permission for a decision already made.

Name the behaviour, not the answer you hope to hear

Ask what a participant must notice, understand or do for either choice to work. In a live squad test, that might be whether first-time players find the shared objective without a teammate explaining it, whether a losing team can identify a recovery option, or whether players coordinate before the first irreversible choice.

Observable behaviour is harder to bias than a leading question. “Was the objective clear?” invites agreement. Watching where players look, what they try, when they ask for help and what they believe happened gives the interview something concrete to examine.

Choose the smallest method that can answer it

Expert review

Use it to find known interaction, accessibility or information-design problems before recruiting players.

Individual usability session

Use it when one person can encounter the critical path without a live team changing the conditions.

Synchronized multiplayer session

Use it when coordination, social pressure, shared timing, role interaction or live opposition creates the behaviour.

Unmoderated or telemetry work

Use it when frequency across a larger sample matters more than understanding the moment in depth.

A live lobby is expensive in attention and scheduling. Use it because the room is part of the question, not because multiplayer footage looks persuasive in a presentation.

Let the decision determine the cohort

Recruit the people who could produce evidence against the current assumption. A group of genre experts cannot tell you whether a convention is discoverable to newcomers. A room of first-time players cannot tell you whether high-level counterplay survives repeated use. Mixing both can be useful, but only when each group remains visible in the record.

Region, hardware, role, experience and whether participants know each other are test conditions. Do not describe them as demographic decoration after the room has been filled.

Write the evidence rule before the room opens

State what would strengthen either option and what would merely be interesting. A useful rule might require the same breakdown to appear independently across several first-time squads, plus a debrief explanation that matches the observed behaviour. One confident comment is not automatically a finding, and a quiet room is not automatically success.

Also state what would invalidate the session: the wrong build, failed voice, a missing role, a route problem or a teammate teaching the exact behaviour under study. Recording these limits protects a negative result from being tidied away and a positive result from being made larger than it is.

A brief that can survive contact with the session

• Decision: the real alternatives and the milestone they affect.
• Population: who must be able to use the design.
• Behaviour: what participants need to notice, understand or do.
• Comparison: the current build, an alternative, or a declared baseline.
• Evidence: what would change confidence in either direction.
• Limits: what this method, cohort and build cannot establish.
• Next action: who makes the decision when the evidence arrives.

If those seven lines are clear, the recruitment, run sheet and discussion guide can be designed around them. If they are not, adding participants usually adds ambiguity rather than confidence.