search

infclass / teeworlds-infclassR

InfClass server
Other
18 stars 18 forks source link

Excessive CPU Usage #79

Closed miguilimzero closed 2 years ago

miguilimzero commented 2 years ago

Current version of infclass consumes huge amount of cpu compared to other mods and compared to yavl infclass. Making it impossible to host the mode on 1 core and 512 mb ram cloud servers, lagging the entire servers and all other mods hosted on it.

Because it is a teeworlds mod and the old version does not have this problem, I believe that this should not be expected and it must be a bug.

7 Mods with no Infclass:

no-infclass

7 Mods + Yavl Infclass:

infclass-yavl

7 Mods + This Infclass:

infclass-kaffeine

Kaffeine commented 2 years ago

Hello. The server codebase comes closer and closer to DDNet. It will be fair to compare the CPU load with another DDNet (or DDNet-based) server with a comparable number of players.

Kaffeine commented 2 years ago

I started the server, joined with two players, and played for a minute. image

The most expansive operation is the map loading — decompression of the server map, copy-pasting of the layers, (insertion of new extra layers for the client,) then compression with SHA256 checksum.

We had some regressions which resulted in much higher CPU usage but it was fixed long ago. Now everything seems to work as expected.

Personally I decided to disable MySQL integration to reduce the server load (now it is disabled by default and probably even broken). I'll continue investigation but I don't see what we can do.

For example, if you'll see that there are spikes in the load during rounds — maybe related to specific features or entity — then there would be something to work on. Right now the only missing load-reduction feature is a cache for server info requests, but it noticeable only when the server is under info-requests attack.

Kaffeine commented 2 years ago

Infclass-specific load, measured during a game round (so this excludes map-related operations):

If we'll optimize all of that top-load by a magnitude, then the total server CPU load will be cut by about 8%

Only CInfClassGameController::GetBlindCharactersMask() and CInfClassGameController::HandleLastHookers() is something new. All other loads exist since necropotame original work (and inherited in yavl fork, and then here).

Kaffeine commented 2 years ago

Because it is a teeworlds mod and the old version does not have this problem, I believe that this should not be expected and it must be a bug.

7 Mods + Yavl Infclass:

7 Mods + This Infclass:

The load depends on the current players. Yavl server was running for 1 minute, this server was running for 12 minutes. How many players had the servers? Or are we talking about a load with 0 players? If it is a "bug", then maybe it is in some specific player class.

E.g. slug class creates 10x more load than a hunter, because the slug slime is very expensive.

miguilimzero commented 2 years ago

@Kaffeine All servers were with no players at the round 0 for all mods. DDNet have the same and in some cases less CPU usage then the other mods, about to 1% with no players.

This Infclass starts using 40% of CPU with no players since minute 0 of the process.

Kaffeine commented 2 years ago

In that case it must be a bug. I had no idea you're about an empty server. I'll investigate.

Kaffeine commented 2 years ago

I found the issue. ThreadPool (IIUC it is used e.g. for registration on masterserver) caused an extra load even when idling. I need a few days to verify that the fix does not break something else (ThreadPool is a very danger place and a smallest mistake might result in a crash).

I'll run this code for three days and If there 'll be no regressions/crashes then I'll release v1.4.1 with this and a few more (important) fixes on March 23.

The fix is pushed to staging branch for now (commit bdd449a305aacb56e546b83b0a4bd8215f484150). Feel free to try it, I would say that 90% that there is no regressions and this code works better than the current production.

miguilimzero commented 2 years ago

What a fast fix! Now its working flawless.

Kaffeine commented 2 years ago

https://github.com/InfectionDust/teeworlds-infclassR/commit/17a92f7ef64e883a85e6f9913d009d5bc43eafda

Kaffeine commented 2 years ago

Released.