All



Objective A: Is the control scheme understood by the player?



Objective B: Do players use and understand the « catch props » feature naturally?

 Do players understand there is a limit of 6 props?

 Do players understand the range to catch props?

 Do players understand how to take props?

 Do players take some props when doesn’t have any?



Objective C: Do players use and understand the « invoke golem » feature

naturally?

 Do players understand range of invocation?

Objectives

 Do players understand the strategic use of the golem?

 Do players uses more the invoke feature than the launch feature?

and research



Objective D: Do players use and understand the « throw » feature naturally?

 Do players launch props to defend himself from close enemy?

questions

 Do players uses the feature?

 Aim: Do players not feel the aim of the throw feature frustrating and/or

unfair?



Objective E: Do players understand the enemy AI behavior correctly?

 Do players understand the AI behavior correctly? (Behavior: The AI always

attack the forge except if they are close to a golem or the player)

 Do players understand they are supposed to protect the forge?

Objective B: Catch props

KPI:

 Do players understand there is a limit of 6 props?

 Obs: Do players keep trying to take object once they reach the limit of props?

 Data: Number of times a player try to get props while is full

 Itw: « How many props can you take? » « What signals you cannot take more props? »

 Do players understand the range to catch props?

 Obs: Do players try to get props way out of range?

 Itw: « Is there a range to catch props? » « What signals you the range to catch props? »

 Do players understand how to take props?

 Obs: Do players catch more than one prop at the time?

 Itw: « How do you take props? »

 Do players take some props when doesn’t have any?

 Obs: Do players take some props when doesn’t have any?

 Itw: « When did choose to get some new props? Why? »

Objective C: Invoke golem

KPI:

 Do players understand range of invocation?

 Obs: Do playerstry to invoke golem outside of the range of invocation?

 Itw: « Is there a limit to the range of invocation? » « If yes, what signify this limit? »

 Do players understand the strategic use of the golem?

 Obs: Do players invoke golem in diverse places to cover all entrance?

 Datas: Do players always have a minimum of 3 golem alive on the map

 Itw: When and why did you choose to you golems?

 Do players use more the invoke feature than the launch feature?

 Datas: Time spent to invoke golem / Time spent to throw prop

 Itw: « What was your strategy to choose when you create golem or throw props? »

Objective D: Throw props

KPI:

 Do players launch props to defend himself from close enemy?

 Obs: What does the player throw props for?

 Data: « What was you strategy when you were throwing props? »

 Do players use the feature?

 Obs: Do players use this feature?

 Data: Time spent throwing props/ minuts

 Aim: Do players not feel the aim of the throw feature frustrating and/or unfair?

 Obs: Do players has sign of frustration after missing a shot?

 Itw: « What do you think about the shooting propsand the aim system? » « Did the precision of

the throw frustrate you? »

Objective E: Enemy AI Behavior

KPI:

 Do players understand the AI behavior correctly?

(AI Behavior: The AI always attack the forge except if they are close to a golem or the

player)

 Itw: Can you explain me what is the enemy behavior? Who does it focus?

 Do players understand they are supposed to protect the forge?

 Itw: What are you supposed to protect?

All

Method and protocol

 6 players - 30 minutes individual sessions

 Type of users:

 FPS players,18-30yo

 6 players (“Eva” Disrupter and/or “Jonathan” Action normie)

 Playsession schedule

 3 minutes introduction

 3 minutes Control playtest

 20 minutes of free playtime

 14 minutes of interview

Interview grid (in French)

Pas du tout

Partiellement

Réponse

Questionnaire

Compris

Commentaire

compris

Combien de props peut tu prendre à la fois ?

Couleur de la bordure

Qu'est-ce qui signale que tu ne peux plus prendre de props ?

Oui

Il y a t'il une distance pour attraper des props?

Absence de signes

Qu'est-ce qui signale cette distance ?

Comment prend tu des props ?

Quand j'en ai plus

Quand est-ce que tu choisis de prendre des props ? Pourquoi ?

Oui

Il y a t'il une distance pour invoquer les golems ?

Disparition du signes

Qu'est-ce qui signale cette distance ?

Quand et pourquoi choisit tu d'invoquer des golems ?

Quelle est ta stratégie pour savoir quand créer des golems et

quand jeter des props ?

Que penses tu du système de visé et du lancement de props ?

Est-ce que le lancement est frustrant ?

Peut tu m'expliquer le comportement des IA ennemies ?

Quel est le but du jeu ?

Comment est-ce tu survis ?

Peut tu me décrire la forge ?

Quelle barre de vie représente quoi ?

Autre ?

Aime ?

Feedback box

 The idea behind the feedback box if to create a better game

environment where the players can still give feedback to the GUR

without interruption of the session and the flow of the game.

Feedback was discussed afterward in the interview.

 When the player press a button (green for « I like », yellow for « I

don’t understand », red for « I don’t like ») it generate a line in the

log file and the color show live on the video.

Restream

 The playsession was restream live with discord to the production team (the

playtesters were informed about it). The idea was that the production team

could see players play the game in good conditions (and not with a jury

member asking questions in a noisy environment).

Method and protocol



6 players - 30 minutes individual sessions

 Type of users:

 FPS players,18-30yo

 6 players (“Eva” Disrupter and/or “Jonathan” Action normie)

 1 player had a build with the wrong number of enemies and could not test the fight and gameplay loop in proper

conditions. The player's results will be kept for basic analysis but ignored for game strategic analysis.

 Playsession schedule



3 minutes introduction



3 minutes Control playtest



20 minutes of free playtime



14 minutes of interview

Objective A: All players understood the

control scheme

Objective B: All players used and understood

the « catch props » feature naturally

Objective C: All players used and understood

Goals

the « invoke props » feature naturally

Results

Objective D: All players understood the «

throw » feature naturally but some players did

not use the « throw » feature

Objective E: Most players understood the AI

behavior naturally

Objective A: All players understood the control scheme

All 6 players fully understood, used and remembered the control scheme

after being told orally 1 time. (The "destroy a golem" key was not part of the

test)

Informative data: How do users take props? Click on each props individually,

Hold to catch multiple props or Both?

Both options are used by the range of users

Player 1

Player 2

Player 3

Player 4

Player 5

Player 6

Click (multiple

Take props

Hold

Both

times)

Objective B: All players use and understood the « catch

props » feature naturally

All 6 players understood the limit of props and the maximum number of props

they could grab(6).

All 6 players understood the limit of props signifiers.

All 6 players understood the range to catch props and its signifiers.

All 6 players always try to grab props. “I am looking to have as many props as

I can”

Issue: The range to catch props is a bit short and did not feel natural to most

players, who had troubles grabbing props due to the range. (see video)

Cause: All game features (environment, speed, range of invocation…)

have big metrics except this one

Impact: The range can cause some signs of frustrations.

Issue: 2 players had troubles grabbing objects due to the precision of the

catch feature

Cause: Some props are really small (book for example), high sensitivity mouse,

and players with desire to catch props really fast (without being precise).

Impact: The precision can cause some signs of frustrations. A challenge is

created in a game feature which is not supposed to be challenging.

Video:

Objective C: All players use and understood the « invoke

props » feature naturally

All 6 players understood the existence of a range to invoke golems and the

signifiers

Issue: But some players felt frustrated and did not understood the purpose of

the range (video)

Cause: The range of invocation is so big, that players only notice the range

when they try to be creative

Impact: Reduce the global strategic depth and innovation (not good for Eva

- The disruptor)

All 5 players (in correct game conditions) understood the strategic aspect of

the golem creation and use them to defend the forge while the look for

more props.

All 5 players (in correct game conditions) used the invoke feature more than

the throw feature.

Issue: Players did not understand their golems could die and that the props

were redistributed when they did. Except 1 player who saw it in front of them.

(video)

Cause: Information is never given to the player. No life signifiers on the

golems.

Impact: Players do not look for props near the Forge after they took them a

first time. Some players experience frustration seeing their golems disappear

without knowing why.

Objective D: All players understood the « throw » feature

naturally but some players did not use the « throw »

feature

Issue: The throw feature is not impactful enough:

2 players, after testing the throw feature, decided to stop using it for the

rest of the play sessions

Others felt frustrated while shooting at enemies and not killing them

3 players expressed not doing enough damage with the throw

Cause: Gameplay dissonance: high impact visual, but low impact gameplay.

The throw push enemy (which make them harder to hit) and use crucial

resources

Impact: Player frustration, players choosing to shoot all their props at once

(and not one at a time) to be impactful with the throw feature.

Issue: 1 player chose to use the throw and not the invoke feature when they

encountered too many enemies.

1 player chose to use the throw to protect the golem they had invoked.

Cause: Hard to understand the impact or the life total of the golems, strong

game feel with the throw feature

Impact: Losing strategy without the player realizing it, player thinking it’s the

intended strategy and they were not “good enough”

The throw feature was rightfully used by players outside of the forge.

No players felt frustrated by the accuracy required by the throw feature

Objective E: Most players understood the AI behavior

naturally

5 players (of the 5 in the right settings) understood that the AI was attacking

the forge, but:

1 out of 5 player did not understand that the enemy would attack them if

they were in range.

1 player thought that they could not change the enemies focus anymore

when in close range of the Forge (and thus couldn't protect it)

3 out of 5 players did not understand the enemy would attack the golems.

Color change attack animation was not understood by any players

Decisions taken

 Delete the range to the « invoke » feature: the range was not seen as

necessery by the team.

 Increase the range of the « catch » feature: the feature is still to watch out

on the next playtests to iterate and find a value which feel natural

 Throw now one-shot enemy but there is more enemies (to keep the golem

more relevant)

 No action taken on the catch precision  Eye to keep on that feature in

the next playtests