Demo: Recorded Messages decission

[obligaron]

Introduction

When we record a demo, we need to record what the player does to replay it. There are multiple alternatives what can be recorded. Each alternative has some pros and cons. Currently UI/Click-based recording is implemented. But we should explicitly decide what alternative (and with them what pro and contra) we want. Switching from one alternative to another is expensive (switch implementation, existing replays gets invalid, ...), so we should explicitly decide what alternative we want to be implemented. This issue can be closed if we have a list of alternatives, a overview of the pro and cons of them and a decision what we want.

Notes to admins: Feel free to edit this post.

Alternatives

UI/Click-based

Description

We record players clicks, keyboard input, mouse movements, ... The replay simulates the same interactions with the UI.

Implementation notes

Cause the UI interactions are replayed, we need to know what UI settings (resolution, panel distance, ...) where used when recording the replay. Else the replay would get out of sync with original play (for example click on the wrong UI buttons cause the panel moved to different location).

Pros

• You can see the original players mouse movements, See how he inspects the inventory or how he interacts with the Griswold.
• Currently implemented 😉
• UI is also tested

Cons

• If the UI changes, replays can get invalid (more unstable).
• Replay with different UI or different keyboard bindings is more work and adds more complexity (for example see #2729).
• Controller support needs to be added (for example controller could also uses messages).
• It's hard(er) to show different content then the original player (apm, inventory, monster statistics).
• Demo format is nearly impossible to read, tweak, fix, or adjust.
• Demo files gets bigger, cause more information (mouse movements etc.) are recorded.

Command-based

Description

We record players commands. For example move to tile xy, attack monster, pick up item, ... The replay simulates the same commands.

Implementation notes

This is similar to how the multiplayer work. It's possible that there are synergies between replay and multiplayer. Cause we only records actions that have a meaning to game logic, we don't record something like mouse movements. But that also means you will never see mouse movements in the replay.

Pros

• Doesn't depend on UI aspects and cause of this is more stable.
• Would help the engine, cause this could be a good base to abstract the difference input types (mouse+keyboard, controller, touch). (Note: this could be done even if we don't take this alternative)
• Could perhaps used also for multiplayer.
• Viewing different content (for example another resolution, apm, inventory, monster statistics) then the original player is easier.

Cons

• You never see original players mouse movements or see how he inspects the inventory or how he interacts with Griswold. It's similar to how you see another player in multiplayer.
• UI is not tested.

Ending

This issue shouldn't hold us back to improve the currently demo. The currently stated pros and cons are only a initial list. I hope we can have a nice discussion of the pros and cons. 🙂

[AJenbo]

For comparison Dooms demo format consist of the players current state (moving in direction X at a velocity of 50), which is more like a command based recording, though arguably it is even one step deeper in the engine.

I do not know the internal format of StarCraft demos, but during replay you do not see the players mouse or UI interactions so it appears to be using a command based recording as well.

One pro I think you missed with Command-based is that it allows you to control what UI info is presented during playback, which can be helpful to better understand what was going on during game play. i.e. show the stats all the time to see exactly how low the players health drops, or turn of the automap to a a beautiful action replay as your live desktop wallpaper. This is also mere like StarCraft where you get to inspect the game during playback and have separate controls for adjusting playback speed (including pause).

[AJenbo]

Technically you could also have a hybrid format that includes mouse movement in addition to commands, but I think that might add more cons then it's worth.

[AJenbo]

Oh, and a con for input based recording is that it is nearly impossible to read, tweak, fix, or adjust, even with tooling as the data has no real context with out the specific engine state.

[AJenbo]

More cons against input based recording :sweat_smile:

Since input is related to the rendered we need to record the exact frame time to be able to calculate the exact meaning, and set the engine to this state during playback. This causes the playback to be extremely jittery at time.

Recording input also results in much larger demo files, a full run of the game takes about 20mb, though compression gets this down to just 2mb, I think that a command based format will be much smaller.

[obligaron]

For comparison Dooms demo format consist of the players current state (moving in direction X at a velocity of 50), which is more like a command based recording, though arguably it is even one step deeper in the engine.

I do not know the internal format of StarCraft demos, but during replay you do not see the players mouse or UI interactions so it appears to be using a command based recording as well.

I checked my version of StarCraft 2. You can change spectator mode to follow the camera of the player. You also see which unites or buildings the player selects. But you don't see players cursor.

A big difference to Doom or StarCraft is that both don't have much different screens/dialogs/menus that you can interact with. That is different in diablo. You have the npcs (with shops), inventory (with belt), quest, character sheet. And these are core game mechanics. We miss this core mechanics if we only record what actions the player effective does. Example: If we don't record what the player the player does, you could see the player standing for 2 minutes in front of a npc and you would see... nothing. Cause we don't replay when he is browsing the shop items, sell items, repair his stuff or rearrange his inventory.

One pro I think you missed with Command-based is that it allows you to control what UI info is presented during playback, which can be helpful to better understand what was going on during game play. i.e. show the stats all the time to see exactly how low the players health drops, or turn of the automap to a a beautiful action replay as your live desktop wallpaper. This is also mere like StarCraft where you get to inspect the game during playback and have separate controls for adjusting playback speed (including pause).

I think this is also possible with UI/Click-based approach, but adds more complexity. That's why I added Viewing different content (for example another resolution) then the original player is easier. I added more examples to the pro and added it explicitly as a contra to the UI/Click-based alternative. 🙂

Oh, and a con for input based recording is that it is nearly impossible to read, tweak, fix, or adjust, even with tooling as the data has no real context with out the specific engine state.

That's true. It's a mess with all the click coordinates. 😄 Added it as contra. 🙂

Since input is related to the rendered we need to record the exact frame time to be able to calculate the exact meaning, and set the engine to this state during playback. This causes the playback to be extremely jittery at time.

I tried to describe this with If the UI changes, replays can get invalid (more unstable).. Two questions:

• Should I rephrase this point or add a extra point (something like depends on exact frame time)?
• Do I miss the point? Do you mean that the replay could also jitter?

Recording input also results in much larger demo files, a full run of the game takes about 20mb, though compression gets this down to just 2mb, I think that a command based format will be much smaller.

I also think so. Added it as a contra. 🙂

[AJenbo]

Do I miss the point? Do you mean that the replay could also jitter?

Currently the rendering will jitter since they are replayed at your current time, but the demo has set a fame time that is likely not visually correct during playback. This can cause the screen to shake a few pixels between each frame, or play at an inconsistent pace (very visible if you slow down the replay to 5 ticks/sec)

[obligaron]

Currently the rendering will jitter since they are replayed at your current time, but the demo has set a fame time that is likely not visually correct during playback. This can cause the screen to shake a few pixels between each frame, or play at an inconsistent pace (very visible if you slow down the replay to 5 ticks/sec)

I think this is a bug (see #2750).

[ThomasChr]

I think the command based version is definitly more stable and has the added bonus that the viewer could open windows to see stats while the demo plays. So the only thing needed is to add the ability to record which interactions with the NPCs happend. But that is for sure more easy then coping with newly added buttons in the UI!

[NiteKat]

Just tossing my two cents in, command based version is what I would vote for. A cool feature in that would be to have the ability to decouple the camera from the player doing the demo - so you can move around the current floor without having to follow the player. If demos are to work in MP eventually, would be cool to see what other players were up to while you were fighting a horde of monsters alone. 😆

[obligaron]

From a technical/architecture view command-based solution is clearly better. And I would also prefer this solution.

But before we close this issue, I would ask a last time if we want some of the missing features to be added additionally to the command-based solution. I heard at least one comment. 😉

Feature	Description
Mouse movement	You would see where the mouse (in dungeon) is for the player.
Opened Panels	You can see what panels (inventory, character, quest) the player has currently open.
Selection	You see what object the player selects (in the dungeon; for example a monster or a shrine).
Inventory Actions	Inventory selection. Item pickup in inventory and held in cursor.
Automap	You can see if the automap of the player is open and what location it is pointed to.
NPC Interaction	Shows what menus the player is currently browsing (shop, talk, etc.).
Messages from objects	You see the same shrine messages as the player. (could be included per default)
Videos	You see the diablo kill movie. 😉 (could be included per default)
...	Something I missed.

Note that all listened features are not included per default in a command-based solution, cause they are not needed for multiplayer and game logic. So additional features add more effort and complexity. That means they are not free and perhaps we will never add them. But if we really want them, it would be great to know beforehand, cause some design decisions of the command-based solution could be affected by these features.

Possible priorities could be

• Not interested.
• If you have too much free time. 😛
• It is okay if it's just in the second or third version of this feature.
• It must be in the first version else demo recording is useless!

All features with no reply would be interpreted as not interested. 😁

[AJenbo]

We shall have a command format that can be used both with network, demo, local and hotseat!

[ThomasChr]

And so it shall be!