search

surftimer / SurfTimer

This repository is a continuation of cksurf, originally developed by jonitaikaponi, sneaK, nikoo77, fluffys. Meant to unite developers, so everybody can host a proper surf server.
https://discord.surftimer.dev
GNU General Public License v3.0
146 stars 72 forks source link

[BUG] Replay Server Crash #131

Open FR33K1LL opened 4 years ago

FR33K1LL commented 4 years ago

Describe the bug This issue has occurred since before 285 for me. The introduction of the memory leak fixes in 285 increased the amount of time before the server actually froze up and crashed from this. The server will freeze and give the red disconnect text in top right. Cant enter anything in server console, have to close and restart sourceds.

[SM] Exception reported: Failed to grow array [SM] Blaming: SurfTimer.smx [SM] Call stack trace: [SM] [0] PushArrayArray [SM] [1] Line 926, surftimer/replay.sp::RecordReplay [SM] [2] Line 1082, surftimer/hooks.sp::OnPlayerRunCmd

accelerator crash log

Expected behavior No crash.

Screenshots If applicable, add screenshots to help explain your problem.

Plugin List

Timer Version dev 285.7

Additional context I had been trying to find the source of this for a while, and I think its a buffer overflow from the replay bots and/or the !replay cmd. Ive noticed it crash more often when i notice ppl using the !replay cmd, but also crash when not using it. It used to take about 8-10 hours to crash but the recent mem leak fixes have increased it to as far as 26 hours. Unfortunately it still can occur anytime between 8-26 hours. On 285.7 ive noticed that sometimes now it wont entirely freeze the server (still can though), but will spam that sm exception reported error in console 5 times a second infinitely, and cause havoc in other ways like not allowing zones to work properly. (surf_unicorn, start zone resets to somewhere in the sky on s6, still runs and sets times, but needs a server restart to fix)

Let me know what else I can do to help.

sneak-it commented 4 years ago

What's your sourcemod version?

After many hours of running the gameserver with many players, run sm_dump_handles handles.txt. In the root /csgo/ directory you'll see handles.txt, upload that here.

FR33K1LL commented 4 years ago

SourceMod Version: 1.10.0.6488

I just had to restart the server, so ill wait till tomorrow to upload the .txt, does it matter if theres been a crash or not. Or is this dump with no crashes..

sneak-it commented 4 years ago

It would only really be useful to get that file closer to the time of a crash, after the server has been up for more time. I'm essentially looking to identify a leaking plugin and narrow down what it is, and this racks up the longer the server has been on and with more players.

How much RAM do you have available/assigned to this CS:GO instance that is crashing?

FR33K1LL commented 4 years ago

Ok, ill try and take a dump every hour after 8 hours to get as close as possible. Ive got 16gb on the machine, no ram is specifically assigned to it.

Heres the process right now with 30 ppl on the server.

image

FR33K1LL commented 4 years ago

and cause havoc in other ways like not allowing zones to work properly. (surf_unicorn, start zone resets to somewhere in the sky on s6, still runs and sets times, but needs a server restart to fix it

i think this is caused by the replay crash causing stripper to fail or something at the same time, which is why the zones messed up on unicorn?

Edit: ^^this is irrelevant

sneak-it commented 4 years ago

Entirely unrelated, this is most likely not even a crash due to replays, the error you are seeing is because memory is being dynamically allocated to the replay array, but cannot allocate any more at the time of the crash.

How much RAM do you have available/assigned to this CS:GO instance that is crashing?

qawery-just-sad commented 4 years ago

similar issues #135 and #54

sneak-it commented 4 years ago

This is unrelated to those 2 issues, but one is a duplicate of the first.

FR33K1LL commented 4 years ago

@sneak-it apologies for the delay man, ive been waiting for a replay crash while periodically taking dumps, but ive yet to see the replay crash because i am now getting map crashes (specific maps?). idk if its because the stripper cfgs were changed, or if its surftimer related.

FR33K1LL commented 4 years ago

ok, finally. i got a dump about an hour before the crash. I couldn't post to paste bin because it was over a mb in size, i found a free alternative though -> https://paste.ee/p/izxEa

I hope this helps, let me know what else i can do.

Bara commented 4 years ago

https://snksrv.com/tools/sourcemod/handles-analyzer.html

Please use this

FR33K1LL commented 4 years ago

heres the output

https://paste.ee/p/FY0IF

olokos commented 4 years ago

Does this still occur for you with 1.0.1? @FR33K1LL

Bara commented 4 years ago

Are you able to reproduce this with the latest dev Version? If you get any errors, please update or post the errors again.

Don't forget after updating surftimer to run the command sm_surftimer_upgrade and after this to .sql-File.