search

airmash-refugees / airmash-games

Game server data used to generate https://data.airmash.online/games and https://starma.sh/games
2 stars 5 forks source link

us-ctf2 server outage #2

Closed congratulatio closed 4 years ago

congratulatio commented 4 years ago

@x-airmash, your server www.airmash.xyz is currently unreachable - it's been down since 2020-02-17 07:52 UTC, according to the games data backend service.

congratulatio commented 4 years ago

@steamroller-airmash the https://starma.sh/games endpoint is timing out, and I think it's because of this issue, as it will be waiting for https://www.airmash.xyz/ctf1 to respond.

Temporary mitigation: removal from the games list in https://github.com/airmash-refugees/airmash-games/commit/c7dbcc5fae4c94c6032d0b5547fe01b2af70d607.

steamroller-airmash commented 4 years ago

https://starma.sh/games is working again now (for me at least). Removal of the affected server seems to have resolved it. Not sure why a server failing to respond caused the starmash backend to hang.

congratulatio commented 4 years ago

The game server seems to be down entirely, and isn't responding to any TCP connections at all, not even to reject them, so the Starmash backend defaulted to the operating system timeout on connection. However, the Cloudflare proxy timed out first, and responded with a 504 error rather than giving the backend long enough to finish its timeout and produce the games JSON.

I've submitted a PR here to mitigate this: https://github.com/steamroller-airmash/airmash-front-page-server/pull/1

congratulatio commented 4 years ago

Closing this issue as it's been over a week since this server went down, and there is no indication that it will return.