becomes increasingly sluggish. After one hour, all players are completely frozen

[zbhcn]

Version on September 19. However, after the server runs for a certain period of time, approximately 15 minutes later, the response time of .server info changes from 10 milliseconds to more than 50 milliseconds and becomes increasingly sluggish. After one hour, all players are completely frozen，but not crash

[EricksOliveira]

I noticed that my Delay started after the Bots' AutoTeleport. So I decided to test by deactivating AiPlayerbot.AutoTeleportForLevel = 0. So far I have almost 6 hours online without Delay.

[noisiver]

I never had any actual delay. After 5 hours it was up to 16GB of RAM usage but no actual lag. It wasn't close to hitting the RAM limit though.

[serverwar]

My setup: 128 GB of RAM, CPU E5-2630 v4 @ 2.20GHz Number of processors: 40 Even using only 30 GB of RAM and 5% of the processor, I experienced delays. I started changing settings and noticed that while I had no delay, someone I was talking with did. They suggested changing AiPlayerbot.AutoTeleportForLevel = 0 When I checked mine, I had already disabled it, and I had no delay. In summary, it's 99.9% certain that this setting causes the delay. After they disabled it and I kept it at 0, I experienced zero delay.

-
Even with this configuration I had delay, until I changed the settings, The problem is not the lack of resources, but what npcbot does within the game.

[serverwar]

Before changing the settings it looked like this

[noisiver]

My setup: 128 GB of RAM, CPU E5-2630 v4 @ 2.20GHz Number of processors: 40 Even using only 30 GB of RAM and 5% of the processor, I experienced delays. I started changing settings and noticed that while I had no delay, someone I was talking with did. They suggested changing AiPlayerbot.AutoTeleportForLevel = 0 When I checked mine, I had already disabled it, and I had no delay. In summary, it's 99.9% certain that this setting causes the delay. After they disabled it and I kept it at 0, I experienced zero delay.

- Even with this configuration I had delay, until I changed the settings, The problem is not the lack of resources, but what npcbot does within the game.

You don't have 40 processors 😆 You really seem to think you can ever use that many cores. Not in a million years. At best you might be able to use 5 cores.

Overall CPU usage is absolutely pointless to look at too, what you need to look at is each individual core. 2.2GHz will not be able to handle much. Even at 3.4GHz I start to get increased update diff times at 2000 bots although not noticeable at that point.

Even if I throw all the cores I have at it it'll still only use up to 5 so the overall CPU usage will show a low number while in fact the cores it's using are maxed out or at least close to maxed out.

Oh and also: Windows gives you much, much worse performance compared to Linux. I mean a seriously huge difference.

[Aecroo]

Late to the Party, but having the same problems here. Running my setup on a Raspberry Pi 5 with 8GB of RAM and an NVMe drive, and that has worked very well for months. After one of the latest updates (around a month ago), I noticed performance drops after some hours.

Performance monitored with Zabbix (around 6 hours, from crash to crash, worldserver.conf crashes if there is a >30s freeze):

Noticeable is that the SQL delete statements drop if the problems occur:

CPU / Memory / Swap:

This is reproducible; I rebuilt the entire server from scratch.

Modules active: Playerbots and AH-Bot

[Loopher]

I am running into the same issue. After 8 hours or so, my server consumes all the available ram and then stops responding to commands.

[Aecroo]

I rebuild the server from scratch again and created a new Database. That worked fine for like 2-3 days but then the performance got worse again.

Can someone give me a tip on how to monitor / debug the long time diffs even further. My Zabbix Dashboard gives a lot of informations right now but i cannot find the cause.

the time diff max percentile always behaves the same shortly before a crash (>30s)

I suspected the AH-Bot module but disabling it made no differences. Also to turning off options for auto gear upgrade for bots and alike did not help.

[hermensbas]

For now just try to restart the server every day. Prolly due running out of memory.

On Fri, Oct 11, 2024, 13:05 Aecroo @.***> wrote:

I rebuild the server from scratch again and created a new Database. That worked fine for like 2-3 days but then the performance got worse again.

Can someone give me a tip on how to monitor / debug the long time diffs even further. My Zabbix Dashboard gives a lot of informations right now but i cannot find the cause. 2024-10-11-12_52_42.png (view on web) https://github.com/user-attachments/assets/4436af69-aaad-4bb7-a992-5ae4edfe4e6e

the time diff max percentile always behaves the same shortly before a crash (>30s) image.17.png (view on web) https://github.com/user-attachments/assets/7ecdc38a-f5b8-4b9b-9e41-d66533d13943

I suspected the AH-Bot module but disabling it made no differences. Also to turning off options for auto gear upgrade for bots and alike did not help.

— Reply to this email directly, view it on GitHub https://github.com/liyunfan1223/mod-playerbots/issues/537#issuecomment-2407175013, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHOUW6R7OIXA4NYHSPNLVQLZ26WI7AVCNFSM6AAAAABOOBULUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBXGE3TKMBRGM . You are receiving this because you commented.Message ID: @.***>

[mtellese]

Has there been any updates regarding this? I have tried the suggested fixes above such as turning off the autoteleport. Something else is that we cannot seem to turn off the random teleporting entirely, and yet we still get severe lag spikes and freezing like above. In spite of turning off the autoteleport for level

[Trus3683]

Has there been any updates regarding this? I have tried the suggested fixes above such as turning off the autoteleport. Something else is that we cannot seem to turn off the random teleporting entirely, and yet we still get severe lag spikes and freezing like above. In spite of turning off the autoteleport for level

Have you been updating? The most recent versions appear to have significantly helped.

[Badgermilk0]

I had huge issues with this from last weekend build, have their bein focused development regarding this issue? I will try to make a new build later using the latest commit and see if it helps.

[noisiver]

Has there been any updates regarding this? I have tried the suggested fixes above such as turning off the autoteleport. Something else is that we cannot seem to turn off the random teleporting entirely, and yet we still get severe lag spikes and freezing like above. In spite of turning off the autoteleport for level

Unfortunately the teleportation happens regardless of the option because that option (as far as I know) only affects the teleport when the bot levels up. The other teleports are on random timers. For the time being it's the best option since the bots can't walk everywhere yet.

I can't speak for the freezing though. I've never seen it happen on my end.

[mtellese]

Apologies for the delay, to answer your question yes, we did update and it got even worse. Now the worldserver refuses to start when the bots are enabled. I even made sure to update the azerothcore branch to match so that it should all be up to date. So it has sadly gotten worse.

[hermensbas]

To much of all in one issue. AloneActivity are default at 100% and smartscale disabled, keep those settings. Make sure the bot amount is reasonble in relation to your hardware. There is mem footprint growth in general and alot if u have to many bots. Behaviours around is known.

Keep monitoring ur memory usage and restart ur server frequently if needed, and tweak your config based on your hardware and usage. And keep posted for future optimizations.

[noisiver]

I actually had the server lock up earlier today. I logged in fine but after a few minutes the server became unresponsive. I'll add and use gdb while running the server in hopes of detecting what it did when it locked up, maybe it could identify the reason.

[hermensbas]

I actually had the server lock up earlier today. I logged in fine but after a few minutes the server became unresponsive. I'll add and use gdb while running the server in hopes of detecting what it did when it locked up, maybe it could identify the reason.

:// How was ur memory usage?

[noisiver]

Worldserver was using 6.2GB and the overall usage was 9.7GB (out of 24GB)

[hermensbas]

hmmz that seems new