Server Crash on Map Change

[SkyMulley]

Follow up reference in #37 . I've noticed this come up quite a lot recently around server owners. I've gone ahead and spent some time trying to replicate the issue and have found a case where the server crashes on map change. It seems to only happen with some addons over others (Eg. DarkRP runs fine, GmodAdminSuite crashes).

Can't re-open issues so had to post a new one, sorrie about that. 32 bit (I believe), Linux Server

[raecyst]

Changing the version to an earlier version fixed it for me, unsure as to which version I used.

[Maldevelops]

Changing to an earlier version doesn't work for more recent addons.

[FredyH]

So you are saying it crashes even with an older version of mysqloo when you change the map?

[SkyMulley]

I'm using an older version of MySQLOO and the crashes do not occur during map changes, however with the latest it does.

[raecyst]

@FredyH Same issue, essentially we had to switch to an older version to allow map changes since we use map changes a lot with our server events. However this meant that ServerGuard would no longer record bans properly.

[FredyH]

This seems to be related to certain addons that might run queries right as the server reloads, could you post what addons you have installed on your server that use mysql?

[raecyst]

We're using Helix as a gamemode, and ServerGuard as an admin system.

[Maldevelops]

This is what happens on change level - https://gyazo.com/09cc9dee7f0dc5807684481a80ff646c

It does look like GmodAdminSuite seems to be contacting the database right before the changelevel and causes the crash to occur.

[FredyH]

Ah this is actually really helpful, it seems the issue is rather that it runs SQL instantly when the server starts after the map change.

[Maldevelops]

So that means that GmodAdminSuite is the offender in this case or is there a possible fix within mysqloo?

[G-Force007]

Confirmed crashes, even with delays added.

I've delayed my mysql queries from running 20-30 seconds after map change, and it still continues to crash. I'd love to continue using 9.5 but because I'm planning on utilising windows 64 beta there is no support there.

[FredyH]

Could you send me exactly the script you ran that makes the server crash every time? That would help me debug this issue immensely.

[G-Force007]

It's basically my gamemodes functionality. I'd have to strip it out and make a single file. I'll work on this one.

(This is not the issue, I updated the types and recompiled, didn't resolve crashes) Going through some of your recent changes I also noticed that some files are out of date: https://github.com/Facepunch/gmod-module-base/blob/development/include/GarrysMod/Lua/Types.h vs https://github.com/FredyH/MySQLOO/blob/master/GmodLUA/include/GarrysMod/Lua/Types.h

[G-Force007]

Minidump crash files indicate below (Windows 64bit beta of GMOD)

Exception Code: 0xC0000005 Exception Information: The thread tried to read from or write to a virtual address for which it does not have the appropriate access.

Unhandled exception at 0x00007FFDED5CE105 (lua_shared.dll) in be42bcd6-5b65-4ed5-9dc0-88668697b64e.dmp: 0xC0000005: Access violation reading location 0x0000000000000008. Unhandled exception at 0x00007FFDF098E047 (lua_shared.dll) in c01f0c58-1098-4029-b300-27e54ea00417.dmp: 0xC0000005: Access violation reading location 0xFFFFFFFFFFFFFFFF. Unhandled exception at 0x00007FFDF098E0A5 (lua_shared.dll) in a5e6ac1b-61dc-4813-9a05-0f9b0d2ec5c4.dmp: 0xC0000005: Access violation reading location 0x0000000000000063.

lua_shared.dll!00007ffded5cf31a()

[willox]

The relevant stack trace to G-Force's crashes is below. It doesn't tell me much other than that somehow the Lua state's garbage collector has been left in an invalid state? I don't know how i'd even go about trying to reproduce this crash with a binary module of my own.

[Inline Frame] lua_shared.dll!gc_marktrace(global_State *)
lua_shared.dll!gc_traverse_trace(global_State * g, GCtrace * T)
lua_shared.dll!propagatemark(global_State * g)
[Inline Frame] lua_shared.dll!gc_onestep(lua_State *)
lua_shared.dll!lj_gc_step(lua_State * L)
<any cause of GC here>

[Heavybob]

Any updates here @FredyH ?

[FredyH]

Hi, I wish I could look into this right now but I am in a hospital at the moment and will probably stay there for quite some time unfortunately (no, not corona). I will maybe try to replicate it but it would probably be better if someone else could look into it and try and get some code that reproducibly crashes on map change.

[Heavybob]

U better not die, we need this shit fixed. Get well soon.

On Fri, Apr 24, 2020 at 7:52 PM FredyH notifications@github.com wrote:

Hi, I wish I could look into this right now but I am in a hospital at the moment and will probably stay there for quite some time unfortunately (no, not corona). I will maybe try to replicate it but it would probably be better if someone else could look into it.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/FredyH/MySQLOO/issues/45#issuecomment-618916633, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCPJG3IKC6LXQ7AMYBM5PTROFOMNANCNFSM4K46D2KQ .

[Heavybob]

U still better not be dead and getting well sooner. Still need fix eeeeeeee.

[FredyH]

Not dead yet, I already made some changes (specifically switching back to dynamic linking), that should hopefully fix these crashes some people are experiencing, however this will only be released for 64 bit as MySQL does not make 32 bit clients anymore. Expect that new release soon.

[Heavybob]

Fantastic. Thanks for not dying.

On Tue, Jun 2, 2020 at 7:59 PM FredyH notifications@github.com wrote:

Not dead yet, I already made some changes (specifically switching back to dynamic linking), that should hopefully fix these crashes some people are experiencing, however this will only be released for 64 bit as MySQL does not make 32 bit clients anymore. Expect that new release soon.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/FredyH/MySQLOO/issues/45#issuecomment-637432499, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCPJG5DIDKHOCLFV5B2FQLRUTERFANCNFSM4K46D2KQ .

[iTzOneLightning]

@FredyH I have the problem on all 3 of my Servers. Wanted to downgrade to 9.5 again, but it doesn't work. So eather way I'm stuck with 9.6 and no mapchanges or nothing. Can you help with the matter of downgrading? I know you're working on a solution for this. Hope you recover soon!

[FredyH]

If you are running on the 64 bit branch of gmod I could use you to test and see if it is fixed in the new version. If you are interested, add me on Steam: https://steamcommunity.com/id/FredyH

[iTzOneLightning]

@FredyH I am! I just added you

[PieKnu]

I'm having the same issues. But like the others I'm not using GmodAdminSuite. I got:

• PointShop2
• AWarn3
• mLogs2

[FredyH]

What operating system?

[PieKnu]

Oh, it's Ubuntu 18.04.4 LTS 64Bit

[Synkstar]

My server has issues to I have blogs and it freezes at sending queries to database. This is on a x64 server also had the same issues on x86

[FredyH]

If it freezes when sending logs that is most likely the fault of blogs, which possibly waits for the query to complete on the main thread

[Synkstar]

Yes it probably is related to blogs. Also thanks for making this module btw very useful.

[FredyH]

According to @Heavybob the crashes have stopped for them since a recent gmod update. Can anyone confirm this?

[PieKnu]

Totally forgot about this thread. But yea, after the last or penultimate GMod update and for me after upgrading & fetching my old TTT Gamemode + changes to the latest one, no crash occurred. (already approx 2 weeks ago)

[FredyH]

Since no one has responded to this with any crashes, I am going to assume that it was indeed a gmod issue. We have not experienced any crashes on the 64 bit branch of gmod in a good while either.

If anyone does still experience this issue, please reopen it.