Uncaught exceptions in socketio events will crash the entire shard

[johnnyapol]

Some of the recent exploits that have affected server uptime have abused bugs in the protocol to trigger an exception on the shard (torn game instance). These uncaught exceptions then proceed to terminate the entire server process.

We could work around this by disconnecting the client and logging the exception via an exception handler; however, we have to consider the following approaches:

• Let the exceptions crash the game so we can bring attention and fix them
• Log exceptions but let the game continue
• Discard exceptions

The third approach is obviously out of the question. The primary concern I have with the second option is there is the potential for the game to be left in an invalid / corrupt state and causing damage to player data. We'll need to evaluate the nature of the code and see if maybe we could selectively guard socketio logic blocks but leave the rest of the shard code unguarded.

[chrisj1]

Could we log the error and try to recover. We could then have some method to notify an admin to view the logs. In addition, we could create a backup of the game state to revert to if needed.

[DamienVesper]

The exploits that are used (by me) to crash the shard are merely modifying anything sent client-side through the socket (bt) such as variables (changing from string to int, etc.) If the variable type is checked before executing code with the input from the client it should put an end to the crashes and exploits. Spammed user instances should also be fixed (simply add a cooldown and it'll take hours to crash the server).

[johnnyapol]

Upcoming update should patch a bit of the recent issues and enforce type protections. I'm going to demote the priority of this issue just because I think server stability should improve from this. I need to work on implementing better logging frameworks before this would really be effective.

[tardisfromtornspace]

The update is working (I don't need to reload to change the language) but for some reason the translation buttons (specifically Esp) don't respond when clicking (I need to type the /#esp manually).

[DamienVesper]

@johnnyapol Ya fixed the closet door, now work on the back door.

[johnnyapol]

I've come to the conclusion that allowing the shard to die is the best decision here. Recent changes to Mira (https://github.com/TornDotSpace/Mira/commit/2522d14685f08b15988da63e6a7b5f7fd30a6c5f) allow for better logging. Having corruption of player data or invalid / degraded game state is possible if we ignore exceptions.