EntityLimitListener high resources usage

[wellnesscookie]

Description Our TPS has been falling down to 8-14, so we have done a timings report and saw next:

Steps to reproduce the behavior:

• We are unaware of this, but I can submit you a full timings report on private. I can PM it to you on discord if you'd like. I'll let you know if I figure out something in the meantime

Some information:

• Entities count (provided by timings): 2500 - 3500

Server Information:

• Database being used: mySQL
• OS: Linux
• Java Version: Java 8
  • BBox /v output:

    [00:49:03] [main/INFO]: [CHAT] Running PAPER 1.15.2.
    [00:49:03] [main/INFO]: [CHAT] BentoBox version: 1.14.1-SNAPSHOT-b1835
    [00:49:03] [main/INFO]: [CHAT] Database: MYSQL
    [00:49:03] [main/INFO]: [CHAT] Loaded Game Worlds:
    [00:49:03] [main/INFO]: [CHAT] skyblock (skyblock): Overworld, [nether_color]Nether, [end_color]End
    [00:49:03] [main/INFO]: [CHAT] Loaded Addons:
    [00:49:03] [main/INFO]: [CHAT] BentoBox-InvSwitcher 0.0.4 (ENABLED)
    [00:49:03] [main/INFO]: [CHAT] Biomes 1.13.0-SNAPSHOT-b180 (ENABLED)
    [00:49:03] [main/INFO]: [CHAT] BSkyBlock 1.14.0 (ENABLED)
    [00:49:03] [main/INFO]: [CHAT] Challenges 0.8.4-SNAPSHOT-b428 (ENABLED)
    [00:49:03] [main/INFO]: [CHAT] Chat 1.1.0-SNAPSHOT-b30 (ENABLED)
    [00:49:03] [main/INFO]: [CHAT] ControlPanel 1.13.0-SNAPSHOT-b36 (ENABLED)
    [00:49:03] [main/INFO]: [CHAT] DimensionalTrees 1.6.1-SNAPSHOT-b81 (ENABLED)
    [00:49:03] [main/INFO]: [CHAT] IslandUpgrade 1.0 (ENABLED)
    [00:49:03] [main/INFO]: [CHAT] Level 2.3.2-SNAPSHOT-b396 (ENABLED)
    [00:49:03] [main/INFO]: [CHAT] Limits 1.14.0 (ENABLED)
    [00:49:03] [main/INFO]: [CHAT] SerbCraftAddon 2.5 (ENABLED)
    [00:49:03] [main/INFO]: [CHAT] Warps 1.10.0 (ENABLED)

Additional context

• We are open to help and jump in for investigation, sharing additional information and testing!

[tastybento]

I need the actual timings report link. PM on discord or send to tastybento at BentoBox.world.

[tastybento]

Share your config.yml for limits. Do you use group limits?

[wellnesscookie]

I have PM-ed both of them to you on Discord, and no, we do not use group limits. Probably as we never seen them implemented due to no new releases of Limits addon.

[tastybento]

I made the check async. Please try this and see if it helps:

Limits-1.14.0-SNAPSHOT-LOCAL.jar.zip

From profiling the time, the entity counting is what is taking the time. It was not possible to speed it up using code, so the only other way is to make it async and if there's a problem, then remove the entity. I tested it and I don't see any async errors, so hopefully it'll be okay.

[wellnesscookie]

People have reported firstly that their Iron Golem farms stopped working. When an IronGolem spawned, it sent them a message that the limit was crossed (even tho it wasn't, it was 1/20) and that the IronGolem disappeared right after.

We have received so many complaints in matter of minutes, we just had to degrade to an older version of Limits addon. We have previously checked console for errors, there was nothing.

[tastybento]

Huh. That's unexpected. Okay, thanks for checking. I'll try and replicate that issue.

[wellnesscookie]

I'm sorry to reping @tastybento , but we wouldn't if we haven't been experiencing low TPS. We have lowered mob-spawn value related settings to try to get understandable TPS for players to play at the cost of joy.

Is there any other way we can assist towards a solution of this issue?

[tastybento]

When you tried it, was it just iron golems that were the issue? I've been busy at work all week but I could look at this again this weekend. The only way to make the load lower is to move the scan to async, so the approach I used should work, but maybe the iron golem thing needs another listener.

[wellnesscookie]

Clueless to say that, because we reacted very fast to complaints and all of them were pointing towards Golems only. I can make some tests on localhost tomorrow, but I cannot guarantee it will act as same as on a production server.

[tastybento]

I found a bug in the previous version that was counting all the world's entities and not just the ones on the island. Limits-1.14.0-SNAPSHOT-LOCAL.jar.zip Please try it.

[wellnesscookie]

@tastybento roger that. We will attempt to update it tonight, in approximately 7h from now. We will let you know as soon as we test it. Thanks!

[tastybento]

There's a bug that I need to fix where it's counting +1, so if the limit is 2, you can only build 1. Let me fix it now.

[tastybento]

Limits-1.14.0-SNAPSHOT-LOCAL.jar.zip @wellnesscookie Use this one. It fixes the count and also if players build golems or snowmen, etc. it'll drop the blocks if they are not allowed.

[wellnesscookie]

@tastybento we have just uploaded it. We are monitoring it at the moment and are unsure if the TPS is fixed as I speak, because we kicked all players and now they are slowly joining in. It's 00:30h our place, so we won't be expecting many by tomorrow.

Limits, as limits show no problems at the moment and there are no complaints. I will let you know when I collect more information.

[tastybento]

Okay, so assuming there are no functional issues (hopefully not because I tested it), then the thing to look for is timings. I profiled the various tasks that run when an entity is spawned and the task that took the longest was counting how many entities are on the island at the spawn. I allow the spawning, but send the count to an async task that runs in another thread. When it is complete, it will either do nothing if there is no issue or remove the extra entity. So, it will be possible to see the entity for a tick or two and then it will disappear if it is not allowed. With entities like snowmen or golems, or wither, the blocks used will drop to the ground. That's what is happening here. I hope that gives some good context.

[wellnesscookie]

Understood. Our SkyBlock server currently has 41 players online and incrementing as time goes, as for the TPS - it is outstanding with a value of 20.00. I will let you know once more in the afternoon, when players number is at peak.

[wellnesscookie]

It seems to be working very better and Limits event is no longer present in our timings report. Thank you!