FredyH / MySQLOO

MySQLOO
GNU Lesser General Public License v2.1
140 stars 55 forks source link

Crashing on Mapchange #37

Closed NickCloudAT closed 4 years ago

NickCloudAT commented 4 years ago

Since the latest release (9.6) my Server always crashes on mapchange (Garrys mod). When I revert back to 9.5 everything works as expected again.

The error I got was "segmentation fault", not more.

FredyH commented 4 years ago

Are you using the 32 or 64 bit version? If you have the -debug option passed to srcds it should log the stack trace of how it crashed in the debug.log file you can find next to the srcds binary. If you do have a stack trace please post it as well, it would make finding the problem a lot easier.

RockonBen commented 4 years ago

I actually was having the exact same issue. I realized what was causing it, and this is critical. I kept removing addons until I had only ULX-related modules running. Turns out, I have your modified UCL that uses MYSQLOO to authenticate via database (https://forums.ulyssesmod.net/index.php?topic=9672.0). I realized it crashed here. Reverted back to 9.5 and it worked.

I don't know if this is something that has to be patched on MYSQLOO or the UCL MySQL, but at least I found out what's causing it.

RockonBen commented 4 years ago

The strangest thing too is that it loads just fine when you start it up. Maybe because the extension gets initialized first on boot-up and later on map change? I don't know. But in my case it was 100% the modified UCL. Nothing else (as of right now, planning to code new modules) is using MYSQLOO in ULX on my installation.

FredyH commented 4 years ago

Could you attach a stack trace from the debug.log file? Otherwise this is going to be really hard to debug unfortunately. On my server (32 bit) I have not gotten any crashes so far.

Heavybob commented 4 years ago

Potentially having issues with latest release also. Experienced a crash a few days ago that occured after installing the latest release, didn't think much of it. Had something odd occurring today where after 20 hours of the server running and being on the current map for 10 hours, some but not all of my sql workers started failing without explanation. Reverting back and will run some tests on a test server.

32 bit, Windows server.

RockonBen commented 4 years ago

My provider doesn't have debug.log (yes I know it sucks), and I know that MySQL-compatible modified ucl.lua was causing it. To replicate it, download ULX, set up your MySQL modified ucl.lua and change to a different map. Will crash every time.

G-Force007 commented 4 years ago

Has caused crashes + randomly disconnecting workers. No dump file is spat out on crash.

I just wish it wouldn't spam the outdated message every 5 minutes. Sticking with 9.5

FredyH commented 4 years ago

Please also tell me what MySQL version you are using and what OS/bit you are running on. As I said I have not experienced any crashes on Linux 32 bit since quite a few days using this version, so it might be some specific usage that is causing it. I will try and replicate it later today/tomorrow using the ULX library mentioned but it might take some time because I am not using ULX and I have to set that all up first. So if anyone has an entry in the debug.log it'd be greatly appreciated.

G-Force007 commented 4 years ago

Our specs:

Windows Server 2016 running srcds under 32bit. We are using MySQL version 5.7 hosted on a Linux server.

We haven't experienced the crash on map change, the crash was completely random after 10 hours uptime. The crash unfortunately did not spit out a mdmp file and srcds exited with code 255. Nothing printed in console log either.

We have a custom script which opens up multiple connections to the MySQL server with reconnect setup, one worker went into an infinite loop of constantly reconnecting at one stage.

Have not had any of these issues with 9.5

Cannot confirm same issue with ULX MySQL as we do not have ULX connected to our database as it uses the built in SQLite file.

RockonBen commented 4 years ago

Well this is odd.

Can't replicate it on my dev server. At all. So who knows what could be causing it. Note, this was always crashing on main server.

I need to update my dev server but that'll have to wait a little bit. Long story. So for now I'm not sure what to do.

@G-Force007 add -condebug to your command line

NickCloudAT commented 4 years ago

I want to add my System specs:

Ubuntu 18.04 64 bit. I use the 32 bit MYSQLOO Version cause 64 bit somehow does not work.

I tried to use -debug but it does not create a debug.log file.. It did create a "core" file.. but IDK what it does.

FredyH commented 4 years ago

My provider doesn't have debug.log (yes I know it sucks), and I know that MySQL-compatible modified ucl.lua was causing it. To replicate it, download ULX, set up your MySQL modified ucl.lua and change to a different map. Will crash every time.

I also downloaded ulx+the ulc file you linked but it works for me. However I am only testing this with myself on the server, so maybe there's something more complicated required to get it to crash.

FredyH commented 4 years ago

I think the problem is because of a mismatch between the mysql client headers and the mysql client library (version 6 vs 8). I have reverted back to the mysql 6 headers and libraries and published a test release.

Please test the new binaries and tell me if the issues are fixed.

RockonBen commented 4 years ago

Just tried, same crash at same time.

FredyH commented 4 years ago

All of the weird disconnecting issues have been fixed so the only thing strange left is the crashes. However, I am fairly confident it should not be MySQLOO's fault (at least not because of the new version).

The current code is almost identical to the one of 9.5, the only difference is some improved error reporting, which would not crash on server restart (and it would result in a lua panic if anything was wrong). Unless you can provide me with a debug.log entry it's going to be really hard to verify what could cause the crash because I have not been able to get it to crash at all on both my main server and development server.

FredyH commented 4 years ago

I'm gonna close this for now, since I still haven't experienced any crashes on map change or otherwise and I am using mysql quite a lot on my server. If anyone has any more information about this, like a debug.log entry, please feel free to reopen it.

SkyMulley commented 4 years ago

Yo , I've noticed this come up quite a lot recently around server owners. I've gone ahead and spent some time trying to replicate the issue and have found a case where the server crashes on map change. It seems to only happen with some addons over others (Eg. DarkRP runs fine, GmodAdminSuite crashes). debug.log