Reconnect disconnected player

[shelbyd]

Is your feature request related to a problem? Please describe. To make a robust networked game, we need to allow players that lose internet connection to reconnect and continue playing the game.

From playing the examples, I cannot reconnect after disconnecting.

Describe the solution you'd like Ideally, the code in the examples would just work with clients that are catching up from far behind.

Likely the best option is for remote players to share the state snapshot and any inputs for the reconnecting player so that player can replay from that state.

Describe alternatives you've considered Alternatively, all inputs can be recorded and a reconnecting player can replay the entire game. This does not require state to be shared, but requires more and more work as the game runs longer. Additionally, all user inputs would have to be stored from the beginning of the game.

[gschup]

What stopped me so far to implement this feature is not only the problem of "catching up", but mostly the synchronization effort in order to propagate the information that a player disconnected and tries to re-join with more than two clients. Currently, if a client loses connection to any other client, the disconnect is propagated and every client disconnects that player. In order to reconnect, you would have to synchronize the clients again and make sure that all clients are on the same page about this.

I will definitely give this some thought!

[shelbyd]

I don't fully understand why the reconnect synchronization is challenging. But that's probably due to lack of familiarity with the crate internals.

I'd imagine we can model the whole system as basically the following structure:

struct Rollback {
  inputs: Vec<PartialInput>,
  states: SparseVec<GameState>,
}

struct PartialInput {
  player_inputs: HashMap<PlayerId, Input>,
}

The common case will be confirmed frames have input for each PlayerId. But if a player disconnects, other clients at some point will assume we will never have input for that player, and "confirm" the frame without that player's input. While that player is disconnected, other clients will shorten how long they'd wait to confirm a frame.

Once the player reconnects, they will simply start providing inputs again and other clients will see that there are inputs and simply have them for future frame advances.

I suppose my approach is assuming that messages from player A to player B would include inputs for player C that A thinks B doesn't have. Which may or may not be the current case. That might be something to consider supporting anyway, as a fully connected graph would scale quadratically in the number of players.

But even if we require a fully connected graph, it shouldn't be terribly hard to include a "Player C is reconnected" message and the receiving client to reestablish the connection. While the connection is establishing, we can continue advancing the frames as if that player is disconnected.

I see a breaking change in that the serialized game states would need to be portable across potentially different architectures/platforms.

We could also consider an option to "pause" the game while a player is disconnected. It would make sense for some games and not others, but that would not require operating with a player disconnected for some amount of time.

However this is supported, it is absolutely critical for a production networked game. I also may implement this myself (once we decide on the appropriate approach).

[janhohenheim]

Any updates on this?

[gschup]

Due to focussing on my studies, I did not start any work related to this feature.

[janhohenheim]

Is there an existing API that can be used to manually implement this at the moment, e.g. connect arbitrary clients to a running session?

[gschup]

Currently, there is not. Reconnecting or joining mid-game requires multiple things which are not implemented:

• all clients have to receive the information of the reconnect
• all clients need to agree on a reconnect attempt
• all clients need to pause until the reconnect is done
• the reconnected client needs the game state, or a complete history of inputs to catch up (who sends this?)
• all clients need to re-establish connection (synchronization in ggrs) with the reconnected client
• only then, gameplay can be continued

Most of these steps require some kind of design decisions to make, which is why I am hesitant.

Does anyone know any P2P games which offer reconnecting? The games I know that have a reconnect feature are all server-based, which makes this process much easier.

[gschup]

An additional note: how this stuff is propagated relies very much on your matchmaking/lobby architecture, which ggrs does not provide. Maybe reconnecting could be done outside of ggrs and afterwards, a new ggrs session could be started?

[janhohenheim]

Aah, I see, thanks.

I know that some P2P anti-cheat protocols run a referee peer that gets a copy of all communication. I have just read the papers and don't know which games actually implement this, but I can imagine that such games can easily let the referee handle the reconnect by acting as the central authority for the game state.

[samcarey]

I'm thinking of making a long-duration game using matchbox/bevy_ggrs, but the inability to recover from problems is making me wonder if it's feasible. It's just too easy for disconnects to happen, especially since a browser tab gets throttled if the user accidentally hides it.

I'd be willing to implement the recovery process outside of GGRS. I'm imagining the following process:

1. All peers detect that a peer has disconnected.
2. All remaining peers pause, access the game state, and save it locally.
3. All peers end the session, reconnect to the matchbox server, and wait until the disconnected peer joins.
4. One of the peers (chosen randomly) broadcasts the state it previously saved.
5. All peers load the state and begin a new session.

Is it possible for a peer to detect the disconnection and react as described? I think that's the only thing preventing this implementation.

EDIT: Ok I got basically got 1 and 3 above working so I can reset the game automatically. Now I just need to restore the state. I did have to patch matchbox_socket so it doesn't crash when someone disconnects.