Server lag spikes in one world caused by Craftbook

[LadyCailinBot]

CRAFTBOOK-2957 - Reported by Iwitrag

Hello,

I have serious troubles - it never happened before I started to use Craftbook - when there are about 40 players on the server, server get serious lags, players are even unable to move, there are block rollbacks etc...

Here are timings which I have made during the lag: http://aikar.co/timings.php?url=7de3e2bf63f3602afd698bd8f79b494a

Here is full plugin list: http://pastebin.com/nPAeu9hs

As you can see, it's craftbook doing this. PVP world is world, where are all mini-games. There is no god, no fly, PVP enabled and players can't build in this world at all. It uses standard minecraft generator, so nothing strange here. There aren't even any craftbook mechs used... so I don't know why it's making those serious lags.

I have tried to remove craftbook and this bug never appeared again.

Can you help me? Using latest CB and spigot.

[LadyCailinBot]

Comment by me4502

Do you have the snow mechanic enabled?

[LadyCailinBot]

Comment by me4502

Actually, the only mechanic that uses that event is Sponge. You don't happen to have a massive sponge sculpture do you? Or maybe some massive water park where water changes directions a lot?

[LadyCailinBot]

Comment by Iwitrag

Snow mechanics are enabled, but this problem happened even before I have enabled it. Sponge is activated true. Players are able to buy sponge and build with it, but in Creative world, not PVP world.

[LadyCailinBot]

Comment by me4502

Sponge is the only part of CB using it, therefore there must be a lot of sponge or rapidly changing water flows in that world, or another plugin is causing the issue tailing CraftBook into it.

[LadyCailinBot]

Comment by Iwitrag

Well I have removed snow and sponge funcionality and I will see if problem occurs again.

Now, I'm experiencing another error - server shut-down itself randomly each 5 minutes, but I think it's spigot fail or something like that - http://pastebin.com/dbzRTN47

[LadyCailinBot]

Comment by me4502

That error can occur if you have a corrupt jar, or you have replaced the spigot jar whilst the server is running.

[LadyCailinBot]

Comment by Iwitrag

It's not corrupted (same size) and I never send files through FTP when server is running. :\

[LadyCailinBot]

Comment by Iwitrag

So, here is all what I did:

• Used minecraft region fixer in all maps
• Upgraded all plugins to latest versions
• Re-uploaded spigot to server
• Disabled snow and sponge mechanics in CraftBook

And everything works well now, 50 players - 19.98 TPS :)

[LadyCailinBot]

Comment by me4502

Snow had nothing to do with it, so you can re-enable that if you want to.

You could maybe try worldguards sponge feature, it's similar just not with as many features, due to that it may be a little less laggy.

[LadyCailinBot]

Comment by Iwitrag

Now I had simillar problems, but there weren't any lags in timings and TPS we're on 20 - but players were unable to move, it was closing chat and making it unplayable.

After 2 minutes it was gone. This happens randomly when there are lot of players online. I have tried removing craftbook and problem was gone, so I think it's craftbook fault, but it will be pretty hard to track or find what's going on, because as I say, there weren't any lags at all, it was just pushing all players back making them unable to move (it also reseted their point of view (view angle)).

[LadyCailinBot]

Comment by me4502

Are you able to run either jVisualVM and WarmRoast, and provide profiling with them? Also, does disabling sponge fix it?

[LadyCailinBot]

Comment by Iwitrag

I have sponge disabled. I cannot run jVisualVM nor WarmRoast because I'm on Minecraft hosting, I don't have direct access...

I've seen nice fake bot script, which is used to stress test, but it is not up to date. I'll wait until they update it and then I will show you... I will spawn 60 players and everyone will get epic lags (making everyone even me unable to move at all) - and then I will disable craftbook /plugman unload craftbook and lags will be gone.

[LadyCailinBot]

Comment by me4502

Hmm, are you able to disable features one by one?

[LadyCailinBot]

Comment by me4502

Wait, closing chat? That can't happen server side. Is it possible this was client side FPS lag?

[LadyCailinBot]

Comment by Iwitrag

When you teleport, it will close your chat. This happens even if you are teleported on the same place you are already standing.

I can recognize TPS and FPS lags. These weren't FPS nor TPS lags... it was teleports or something like this (it looked like server lags)...

Well, if you know plugin Authme, it was just like when you are not logged in and tried to move, it knockbacked all players back, closing the chat and reseting view angle.

[LadyCailinBot]

Comment by me4502

If there is no TPS lag, it's most likely some other plugin doing it. CraftBook doesn't set positions like that.. But as for why removing craftbook fixes it, I don't know.

Does the server feel like its lagging? The TPS system you're using may be bugged.

[LadyCailinBot]

Comment by Iwitrag

As I said, I have made even timings when the bug occured http://aikar.co/timings.php?url=ea88f5f84bfd5b3e25f0578548475673

Average TPS: 19.94

I have tried unloading HeavySpleef, then BossMessage and then CraftBook... bugs we're gone, next time when those bugs occured I only unloaded CraftBook and again - bugs gone.

I don't really have an idea why Craftbook is doing this on my server, because no other servers have these troubles... It's hard to test it because I need lot of players on server (~45-60) and this bug occurs pure randomly making it hard to simulate.

[LadyCailinBot]

Comment by me4502

Are you able to disable craftbook mechanics one by one, wait until it occurs again, then disable another? And once it hasn't occured for a day or two, message back here with the last one you disabled? This will assist with narrowing it down to a specific mechanic.

[LadyCailinBot]

Comment by Iwitrag

I'll give it a try! Thank you for that idea :) Actually I'm using 8 mech's

1. Custom crafting recipes
2. Head drops
3. GATE
4. BRIDGE
5. LIFT (elevators)
6. Chairs
7. Chunk anchors
8. XP storer

[LadyCailinBot]

Comment by me4502

Maybe disable them in this order

Chairs -> Custom Crafting -> Head Drops -> Lift -> And then any order from there.

[LadyCailinBot]

Comment by Iwitrag

Okay I have informed on our webpage, facebook twitter and in-game that tomorrow we will do stress test on our server, so I hope I will try disabling all 8 mechs. I will follow your order and I will inform you.

Thank you very much for your time and willingness to help me :)

[LadyCailinBot]

Comment by me4502

You're welcome

You may also want to update ProtocolLib to the latest first, as if it is chairs, it may have something to do with it.

[LadyCailinBot]

Comment by Iwitrag

Okay so we did the stress test and it seems that Chairs were doing it. We got epic bugs, then I have disabled chairs in config, reloaded craftbook with plugman and problem is gone.

[LadyCailinBot]

Comment by me4502

Are you able to update ProtocolLib to the latest? It's most likely a bug in ProtocolLib caused by the usage of it with chairs, than craftbooks chairs as they don't really touch that sort of thing.

[LadyCailinBot]

Comment by Iwitrag

Well, I'm using 3.2.0 and it's the latest version (or are there dev builds somewhere?)

[LadyCailinBot]

Comment by me4502

There are dev builds, I'll try finding the link

http://assets.comphenix.net/job/ProtocolLib/

All craftbook does regarding packets and chairs, is it filters out invalid packets the client sends, if the player is currently sitting in a chair.