search

pandorabox-io / pandorabox.io

Pandorabox infrastructure code
https://pandorabox.io
31 stars 4 forks source link

Mesecons issue #271

Closed thomasrudin closed 5 years ago

thomasrudin commented 5 years ago

Situation

Currently we have some mesecons issues, namely lag whenever they are used much. No idea if it is abuse or just the mesecons mod that is not multiplayer capable.

NOTE: mesecons are currently disabled until further notice (2019-08-11)

Details

Whenever a lot of mesecons are triggered, the load of the server increases a bit (but not excessively):

Mesecons-usage (2019-08-10 16:00 to 16:30):

Source: https://monitoring.minetest.land/d/LjYzPumWz/mesecons?orgId=1&from=1565445603933&to=1565447384524

Here the lag stats:

Source: https://monitoring.minetest.land/d/YUpouLmWk/overview?orgId=1&from=1565445603933&to=1565447384524

Interestingly: the lag does not increase a lot on the server side during high mesecons usage. But the interaction to the server grinds to a halt (feels like lag).

This is due to the minetest-server not accepting any incoming packets during that time:

Graphs from the Receive() function of the engine (engine-stats-page: https://pandorabox.io/grafana)

Source: https://pandorabox.io/grafana/d/ifuy9KSZz/minetest-engine?orgId=1&from=1565445603933&to=1565447384524

Somehow the server does not have time anymore to process those incoming packets (interactions, eat, dig, place, etc). This leads to "felt lag" of multiple seconds up to minutes!

Mitigations

I tried with per-mapblock penalties for excessive mapblock use: https://github.com/thomasrudin-mt/monitoring_mesecons/blob/master/init.lua#L21

As that did not really work i decided to stop the mesecons processing entirely.

Next steps

thomasrudin commented 5 years ago

closing, should be fixed now

thomasrudin commented 5 years ago

Not fixed, added debug mod: https://github.com/thomasrudin-mt/mesecons_debug

thomasrudin commented 5 years ago

Changed minetest.conf:

# previous: 40
max_simultaneous_block_sends_per_client = 10
thomasrudin commented 5 years ago

continuing here #309