search

SmartlyDressedGames / Unturned-3.x-Community

Community portion of the Unturned-3.x repo. If you have access to the source code you can find it here:
https://github.com/SmartlyDressedGames/Unturned-3.x/
88 stars 18 forks source link

New Linux Crashes #2088

Closed JustOneMoreBlock closed 1 year ago

JustOneMoreBlock commented 3 years ago

I've tried searching before posting and unable to find any results with a similar error. We have had numerous crashes for days since the last update and there were no logs being outputted.

I started up two servers running pve and they ran for a little bit.

Unexpected mark stack overflow
Caught fatal signal - signo:6 code:-6 errno:0 addr:0x6c8d
Obtained 19 stack frames.
#0  0x007fb2d7268980 in funlockfile
#1  0x007fb2d6ea3fb7 in gsignal
#2  0x007fb2d6ea5921 in abort
#3  0x007fb2d6513146 in GC_push_all
#4  0x007fb2d650884e in monoeg_g_hash_table_foreach
#5  0x007fb2d64f1f19 in mono_unity_liveness_calculation_end
#6  0x007fb2d651a378 in GC_mark_some
#7  0x007fb2d651b4cb in GC_collect_a_little_inner
#8  0x007fb2d651bfb5 in GC_collect_a_little
#9  0x007fb2d846d901 in std::vector<unsigned int, std::allocator<unsigned int> >::_M_fill_insert(__gnu_cxx::__normal_iterator<unsigned int*, std::vector<unsigned int, std::allocator<unsigned int> > >, unsigned long, unsigned int const&)
#10 0x007fb2d8306ab2 in int* std::_V2::__rotate<int*>(int*, int*, int*, std::random_access_iterator_tag)
#11 0x007fb2d8306c46 in int* std::_V2::__rotate<int*>(int*, int*, int*, std::random_access_iterator_tag)
#12 0x007fb2d8366ee1 in int* std::_V2::__rotate<int*>(int*, int*, int*, std::random_access_iterator_tag)
#13 0x007fb2d836134e in int* std::_V2::__rotate<int*>(int*, int*, int*, std::random_access_iterator_tag)
#14 0x007fb2d8361391 in int* std::_V2::__rotate<int*>(int*, int*, int*, std::random_access_iterator_tag)
#15 0x007fb2d83618aa in int* std::_V2::__rotate<int*>(int*, int*, int*, std::random_access_iterator_tag)
#16 0x007fb2d84c9e4e in PlayerMain(int, char**)
#17 0x007fb2d6e86bf7 in __libc_start_main
#18 0x0055715eac8699 in _init
./ServerHelper.sh: line 29: 27789 Aborted                 (core dumped) ./Unturned_Headless.x86_64 -batchmode -nographics "$@"

Both have similar errors

root@ut:/home/Games/scripts# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.5 LTS
Release:        18.04
Codename:       bionic

Startup

screen -dmS ${server} ./ServerHelper.sh +InternetServer/${server}
SDGNelson commented 3 years ago

It looks like a Unity internal crash unfortunately. I will update from 10f1 to 14f1 for this week's update, and upload on the preview branch later today. Hopefully it might have been fixed between those...

SDGNelson commented 3 years ago

Preview branch has been updated now

TH3AL3X commented 3 years ago

I got those problems because i was using a edited code of ServerHelper, so i removed it and is working fine now

gegtor commented 3 years ago

The issue still persists with the same output

For some reason, Russia crashes the most Washington is second

JustOneMoreBlock commented 3 years ago 
Unexpected mark stack overflow
Caught fatal signal - signo:6 code:-6 errno:0 addr:0x4481
Obtained 19 stack frames.
#0  0x007fb07449c980 in funlockfile
#1  0x007fb0740d7fb7 in gsignal
#2  0x007fb0740d9921 in abort
#3  0x007fb0737476c6 in GC_push_all
#4  0x007fb07373cdce in monoeg_g_hash_table_foreach
#5  0x007fb073726499 in mono_unity_liveness_calculation_end
#6  0x007fb07374e8f8 in GC_mark_some
#7  0x007fb07374fa4b in GC_collect_a_little_inner
#8  0x007fb073750535 in GC_collect_a_little
#9  0x007fb0756a2841 in std::vector<unsigned int, std::allocator<unsigned int> >::_M_fill_insert(__gnu_cxx::__normal_iterator<unsigned int*, std::vector<unsigned int, std::allocator<unsigned int> > >, unsigned long, unsigned int const&)
#10 0x007fb07553b852 in int* std::_V2::__rotate<int*>(int*, int*, int*, std::random_access_iterator_tag)
#11 0x007fb07553b9e6 in int* std::_V2::__rotate<int*>(int*, int*, int*, std::random_access_iterator_tag)
#12 0x007fb07559bcb1 in int* std::_V2::__rotate<int*>(int*, int*, int*, std::random_access_iterator_tag)
#13 0x007fb0755960fe in int* std::_V2::__rotate<int*>(int*, int*, int*, std::random_access_iterator_tag)
#14 0x007fb075596141 in int* std::_V2::__rotate<int*>(int*, int*, int*, std::random_access_iterator_tag)
#15 0x007fb07559667a in int* std::_V2::__rotate<int*>(int*, int*, int*, std::random_access_iterator_tag)
#16 0x007fb0756fef3e in PlayerMain(int, char**)
#17 0x007fb0740babf7 in __libc_start_main
#18 0x0056332bb85699 in _init
./ServerHelper.sh: line 29: 17537 Aborted                 (core dumped) ./Unturned_Headless.x86_64 -batchmode -nographics "$@"

Similar errors

I got those problems because i was using a edited code of ServerHelper, so i removed it and is working fine now

Well, the ServerHelper.sh gets automatically updated on update and overwrites the files if they're any changes. This is unmodified anyway. So, this isn't the issue.

TH3AL3X commented 3 years ago

But have you tried it? Because that appears to me when using a modified version...

Freenex1911 commented 3 years ago

I also get the "Unexpected mark stack overflow" crashes on Windows since last update I guess.

ghost commented 3 years ago

It's really strange, had it happen on a few servers too on Windows.

Not sure if helps:

Fault bucket 1758192789342174820, type 5 Event Name: BEX64 Response: Not available Cab Id: 0

Problem signature: P1: Unturned.exe P2: 2019.4.14.14309 P3: 5fa36e85 P4: StackHash_ac46 P5: 0.0.0.0 P6: 00000000 P7: PCH_E7_FROM_ntdll+0x000000000009FCD4 P8: c0000005 P9: 0000000000000008 P10:

Report.wer.zip

In the past 3 days it's happened 3 times on Unturned Rust Modded Server, once on Unturnov 1.5, and once to a Russia map server. Odd, and it seems that it's just internal unity stuff. Sucks to have issues like these, and there's nothing interesting on any of the servers either. None of them even use or have the harmony lib/patch any unturned methods.

TH3AL3X commented 3 years ago

I also get the "Unexpected mark stack overflow" crashes on Windows since last update I guess.

That's a memory leak or what

SDGNelson commented 3 years ago

From reading online it seems to have popped up in other Unity games, and is related to multi-threading and/or garbage collection.

I wonder if anyone with a Windows server has gotten a crash.dmp from this crash? If so we can see whether it is directly related to anything in the game, or if it should be submitted to Unity.

gegtor commented 3 years ago

Can I help with anything if I have Linux?

SDGNelson commented 3 years ago

If you have a .gdb crash dump from Linux it might be helpful. I do not have as much experience with Linux debugging however.

gegtor commented 3 years ago

Linux does not produce .gdb files unfortunately

Freenex1911 commented 3 years ago

Well, I got this error log: error.log

gegtor commented 3 years ago

@SDGNelson This bug should have much bigger priority it made a lot of my players leave my network forever :<

SDGNelson commented 3 years ago

When there were crashes with mono in Unity 2018 I started bundling mono from Unity 2019 with the game, so it might be worth doing a similar experiment using mono from Unity 2020

SDGNelson commented 3 years ago

Testing on Windows with mono from Unity 2020 it seems to work normally, so here are the files if anyone else would like to try:

Windows.zip Linux.zip

On Windows replace the files in MonoBleedingEdge/EmbedRuntime, or on Linux the files in Unturned_Headless_Data/MonoBleedingEdge/x86_64

gegtor commented 3 years ago

So far so good 12+ hours without a crash on Linux

Need a solid day or two more to be sure

SDGNelson commented 3 years ago

Fingers crossed! 🤞 If it works well over the next few days then I will use 2020.1.15f1 mono in the coming Friday update.

gegtor commented 3 years ago

It just crashed :<

Same details as above

TH3AL3X commented 3 years ago

I'm not having any issue with Ubuntu 20.04, i told you is something related of your own scripts, use the default one and you will not get errors....

gegtor commented 3 years ago

This can't in any way be related to scripts all they do is tell the server to start

Also, only this update caused issues so this is pretty logical that there is something wrong I run 8 servers for 2 years now so this issue is very apparent You are not helping...

Freenex1911 commented 3 years ago

Still getting the overflow error on Windows with the new binaries.

SDGNelson commented 3 years ago

Thanks for trying with the 2020 binaries. I'm not sure there is anything else we can do for the meantime except try to mitigate the effects, unless anyone has any ideas. I've been intending to create an external tool for restarting/updating Unturned and Unturned II servers, so I could make that a high priority if it would be useful, but I'm guessing you already have scripts for auto-restarting.

Freenex1911 commented 3 years ago

I guess everyone has restart scripts or appropriate plugins to restart a server automatically. Is there a tool to inspect or debug Unturned? Or is there a way to show detailed RAM usage per thread or per method?

SDGNelson commented 3 years ago

@Freenex1911 To profile/debug the release game, not currently. I have been thinking about moving the tools and asset validation into a separate app in Unity "development build" mode which would allow that. Anyway unless you are seeing unusual memory usage I do not think this is game memory related - we would be seeing a regular Stack Overflow Exception or Out of Memory Exception in those cases.

Freenex1911 commented 3 years ago

Right. But it's weird that the server crashes happen the longer you run your server without a restart.

gegtor commented 3 years ago

Can someone suggest me restart script? I just run my servers in screen with the helper script

Also, I did not found any memory issues or correlation to time when the server was restarted

mcdiggle commented 3 years ago

0 crashes and I have not restarted my servers for like 8 days

JustOneMoreBlock commented 3 years ago

Can someone suggest me restart script? I just run my servers in screen with the helper script

Also, I did not found any memory issues or correlation to time when the server was restarted

Interesting...I run screen too and also get crashes. Might have to move to tmux.

0 crashes and I have not restarted my servers for like 8 days

What OS? Are you using RocketMod?

gegtor commented 3 years ago

This also can't be related to screen or tmux they just handle the sessions

gegtor commented 3 years ago

@JustOneMoreBlock @mcdiggle @Freenex1911 @TH3AL3X

Everyone post what are you using so we might correlate things better OS: Rocket or OpenMod: Libraries: Amount of RAM: Map:

For me it's OS: Ubuntu 18.04 Rocket or OpenMod: OpenMod with Rocket module Libraries: lib32stdc++6 mono-runtime mono-reference-assemblies-2.0 libc6 libgl1-mesa-glx libxcursor1 libxrandr2 libc6-dev libgcc-4.8-dev Amount of RAM: 16GB Map: Russia(Crashes 2/3 per day) Washington (1/2 per day) PEI(No crash ever recorded) As you can see ^ there might be something going on with how much server is taxed by the map and how often it crashes

JustOneMoreBlock commented 3 years ago

I left open CONSOLE and this was the output on most servers. Including PEI.

Unexpected mark stack overflow
Caught fatal signal - signo:6 code:-6 errno:0 addr:0x509c
Obtained 19 stack frames.
#0  0x007f275278e980 in funlockfile
#1  0x007f27523c9fb7 in gsignal
#2  0x007f27523cb921 in abort
#3  0x007f2751a396c6 in GC_push_all
#4  0x007f2751a2edce in monoeg_g_hash_table_foreach
#5  0x007f2751a18499 in mono_unity_liveness_calculation_end
#6  0x007f2751a408f8 in GC_mark_some
#7  0x007f2751a41a4b in GC_collect_a_little_inner
#8  0x007f2751a42535 in GC_collect_a_little
#9  0x007f2753994841 in std::vector<unsigned int, std::allocator<unsigned int> >::_M_fill_insert(__gnu_cxx::__normal_iterator<unsigned int*, std::vector<unsigned int, std::allocator<unsigned int> > >, unsigned long, unsigned int const&)
#10 0x007f275382d852 in int* std::_V2::__rotate<int*>(int*, int*, int*, std::random_access_iterator_tag)
#11 0x007f275382d9e6 in int* std::_V2::__rotate<int*>(int*, int*, int*, std::random_access_iterator_tag)
#12 0x007f275388dcb1 in int* std::_V2::__rotate<int*>(int*, int*, int*, std::random_access_iterator_tag)
#13 0x007f27538880fe in int* std::_V2::__rotate<int*>(int*, int*, int*, std::random_access_iterator_tag)
#14 0x007f2753888141 in int* std::_V2::__rotate<int*>(int*, int*, int*, std::random_access_iterator_tag)
#15 0x007f275388867a in int* std::_V2::__rotate<int*>(int*, int*, int*, std::random_access_iterator_tag)
#16 0x007f27539f0f3e in PlayerMain(int, char**)
#17 0x007f27523acbf7 in __libc_start_main
#18 0x005601c33d0699 in _init
./ServerHelper.sh: line 29: 20636 Aborted                 (core dumped) ./Unturned_Headless.x86_64 -batchmode -nographics "$@"
Ubuntu 18.04.5 LTS
Mono JIT compiler version 4.6.2 (Debian 4.6.2.7+dfsg-1ubuntu1)
RocketMod
RAM: 32 GB
Map: Looks like everything is crashing but Areanas. So, might be logical it could be a plugin issue. So, here's to testing

Plugins to compare

GlobalBan.dll      SDPlugins_Owner_Checker_.dll  Uconomy.dll      Votifier.dll               ZaupFeast.dll        ZaupShop.dll               Zaup_Warning.dll
PlayerInfoLib.dll  TeleportUtil.dll              uEssentials.dll  VotifierRewardUconomy.dll  ZaupLeaderboard.dll  ZaupUconomyEssentials.dll
root@ut:/home/Games/unturned/Servers/pve1/Rocket/Plugins# md5sum *.dll
672e142f6d7e107137ace8a2f1d03929  GlobalBan.dll
213d3136dc523db30db01b2e3be72a8a  PlayerInfoLib.dll
dac858ffc10a520b37fc63da4a3fddb1  SDPlugins_Owner_Checker_.dll
2d8e9ac8e2e436d23a0aff3735c74f59  TeleportUtil.dll
781919233caf33c0e84ba195c67919a7  Uconomy.dll
69b817cf1d89c8c56a3464412c16f5ce  uEssentials.dll
e11ec3483eb9fdf9df76d77878e3757e  Votifier.dll
d4d86d83c75da2cbcc735007f586e58e  VotifierRewardUconomy.dll
2748cd105a466fc3a22782e8e0c9de64  ZaupFeast.dll
49d61703b49b70d8bee9d4cef555c0e3  ZaupLeaderboard.dll
f41256844ede08332568b2eeff3451b5  ZaupShop.dll
fdadc9db7732e2317d8715948b492c9d  ZaupUconomyEssentials.dll
7394e5748fd4587788ba4162020a4489  Zaup_Warning.dll

If you're getting crashes too with OpenMod, and using RocketMod plugins. There could be some kind of correlation that could pin down.

TH3AL3X commented 3 years ago

@JustOneMoreBlock @mcdiggle @Freenex1911 @TH3AL3X

Everyone post what are you using so we might correlate things better OS: Rocket or OpenMod: Libraries: Amount of RAM: Map:

For me it's OS: Ubuntu 18.04 Rocket or OpenMod: OpenMod with Rocket module Libraries: lib32stdc++6 mono-runtime mono-reference-assemblies-2.0 libc6 libgl1-mesa-glx libxcursor1 libxrandr2 libc6-dev libgcc-4.8-dev Amount of RAM: 16GB Map: Russia(Crashes 2/3 per day) Washington (1/2 per day) PEI(No crash ever recorded) As you can see ^ there might be something going on with how much server is taxed by the map and how often it crashes

Are you starting 2 servers in the same instance?

Trojaner commented 3 years ago

I am sure this is caused by one or more bad RocketMod plugins allocating too much stack..

gegtor commented 3 years ago

My OpenMod plugins:

senpai@vps:~/Unturned_Server/Servers/SV2/OpenMod/plugins$ ls -la
total 56
drwxrwxr-x 7 senpai senpai  4096 Nov 29 04:00 .
drwxrwxr-x 4 senpai senpai  4096 Oct 22 22:48 ..
-rw-rw-r-- 1 senpai senpai 28160 Nov 29 04:00 DeathMessenger.dll
drwxrwxr-x 2 senpai senpai  4096 Oct 25 15:50 Pustalorc.GlobalBan
drwxrwxr-x 2 senpai senpai  4096 Oct 25 16:22 Pustalorc.PlayerInfoLib.Unturned
drwxrwxr-x 2 senpai senpai  4096 Oct 23 00:30 Rocket.PermissionLink
drwxrwxr-x 2 senpai senpai  4096 Oct 22 22:48 Rocket.Unturned
drwxrwxr-x 2 senpai senpai  4096 Nov 29 04:00 Tortellio.DeathMessenger

Rocket plugins:

senpai@vps:~/Unturned_Server/Servers/SV2/Rocket/Plugins$ md5sum *.dll
75dfd8d5be60146ae1ddc9b86035791d  ImperialPluginsLoader.dll
e19bef9a8a8b55605a86a9c35c16fe71  JoinLeaveMessages.dll
7684eb4f9396b67ef5b8de6cac9d0918  RichAnnouncer.dll
433f626425b886297599aeecc5425c19  RocketRegions.dll
d90820f60e20e4db263b8fc725cc8c0c  TebexUnturned.dll
9a88e2b1854157de2245fe1972ee00bb  Teleportation.dll
5f900fb49c6eea7da93b393d27b0cee6  VoteRewards.dll
3c4fd95035fc42b362438f4e37ed73d1  WreckingBall.dll
2c9f7df72b5bd53652bc959da13d2a37  Yeeti Loader.dll
69b817cf1d89c8c56a3464412c16f5ce  uEssentials.dll
Freenex1911 commented 3 years ago

I have not changed a plugin in the last months. Suddenly the crashes happened after an Unturned update.

Trojaner commented 3 years ago

Do crashes happen on vanilla Unturned too? With no RocketMod/OpenMod or plugins at all?

gegtor commented 3 years ago

Hard to tell because practically no one runs bare servers today xd Even vanilla ones are rocket based etc

DronXa commented 3 years ago

I just realized, that my semi-vanilla server (with plugins) havent crashed, might be that some plugins is causing this issue.

JustOneMoreBlock commented 3 years ago

The only plugin @gegtor and I use that stood out was uEssentials, however, I disabled this plugin and I still got the same error. So, probably more along the lines of a Libraries thing.

I really wish the Libraries folder was one folder, and then all servers/plugins can access them. Then a Libraries pack should be provided of the most recommended.

In any case, I did find something weird. As it was mentioned that the command line perhaps could have been wrong too.

screen -dmS ${server} ./ServerHelper.sh +InternetServer/${server}

I only bring it up because in my unturned folder. I found a new folder called +InternetServer and inside their are files.

root@ut:/home/Games/unturned# stat +InternetServer/
  File: +InternetServer/
  Size: 4096            Blocks: 8          IO Block: 4096   directory
Device: 903h/2307d      Inode: 526916      Links: 2
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2020-12-02 19:49:21.376135877 -0500
Modify: 2020-11-17 09:48:17.531387228 -0500
Change: 2020-11-17 09:48:17.531387228 -0500
 Birth: -
root@ut:/home/Games/unturned# ls --format=single +InternetServer/
arena1
arena2
arena3
arena4
pve1
pve2
pve3
pve4
pve5
pve6
pve7
pve8
pvp1
root@ut:/home/Games/unturned/+InternetServer# ll
total 420
drwxr-xr-x  2 root root  4096 Nov 17 09:48 ./
drwxr-xr-x 20 root root  4096 Dec  3 05:49 ../
-rw-r--r--  1 root root 36552 Nov 17 09:48 arena1
-rw-r--r--  1 root root 36552 Nov 17 09:48 arena2
-rw-r--r--  1 root root 36287 Nov 17 09:47 arena3
-rw-r--r--  1 root root 36268 Nov 17 09:48 arena4
-rw-r--r--  1 root root 36101 Nov 17 09:48 pve1
-rw-r--r--  1 root root 35977 Nov 17 09:48 pve2
-rw-r--r--  1 root root 35767 Nov 17 09:48 pve3
-rw-r--r--  1 root root 35945 Nov 17 09:48 pve4
-rw-r--r--  1 root root 35526 Nov 17 09:47 pve5
-rw-r--r--  1 root root 37074 Nov 17 09:48 pve6
-rw-r--r--  1 root root 35521 Nov 17 09:48 pve7
-rw-r--r--  1 root root  4712 Nov 17 09:48 pve8
-rw-r--r--  1 root root  1685 Nov 17 09:48 pvp1

They just log files of the server. Although, they haven't been updated. So, probably not the cause unless shell script is reading that folder and causing a conflict in some way.

For now, I'm testing with +secureserver.

JustOneMoreBlock commented 3 years ago 
Linking spawns...
Loading miscellaneous...
dlopen failed trying to load:
steamclient.so
with error:
steamclient.so: wrong ELF class: ELFCLASS32
dlopen failed trying to load:
/root/.steam/sdk64/steamclient.so
with error:
/root/.steam/sdk64/steamclient.so: cannot open shared object file: No such file or directory
[S_API] SteamAPI_Init(): Sys_LoadModule failed to load: /root/.steam/sdk64/steamclient.so
Setting up 4 worker threads for Enlighten.
  Thread -> id: 7efba08b9700 -> priority: 1 
  Thread -> id: 7efb4ffff700 -> priority: 1 
  Thread -> id: 7efb4f7fe700 -> priority: 1 
  Thread -> id: 7efb4effd700 -> priority: 1

Loading Vanilla (No RocketMod)

Setting up 4 worker threads for Enlighten.
  Thread -> id: 7f7442b34700 -> priority: 1 
  Thread -> id: 7f7442333700 -> priority: 1 
  Thread -> id: 7f7441b32700 -> priority: 1 
  Thread -> id: 7f7441331700 -> priority: 1

With RocketMod

Fix:

ln -s /home/Games/steamcmd/linux64/steamclient.so /root/.steam/sdk64/steamclient.so

However, time to start up some games, and they all instantly crash!


Linking spawns...
Loading miscellaneous...
[S_API] SteamAPI_Init(): Loaded local 'steamclient.so' OK.
Caught fatal signal - signo:11 code:1 errno:0 addr:0x2c1c
Obtained 14 stack frames.
#0  0x007fce61f23980 in funlockfile
#1  0x007fce646518ff in _dl_rtld_di_serinfo
#2  0x007fce646523bf in _dl_rtld_di_serinfo
#3  0x007fce61c86cb6 in _dl_sym
#4  0x007fce6191d0e4 in dlclose
#5  0x007fce61c871ef in _dl_catch_exception
#6  0x007fce61c8727f in _dl_catch_error
#7  0x007fce6191d745 in dlerror
#8  0x007fce6191d166 in dlsym
#9  0x007fce6077a1e8 in SteamAPI_GetSteamInstallPath
#10 0x007fce6077a57b in SteamAPI_GetSteamInstallPath
#11 0x007fce6077aed6 in SteamInternal_GameServer_Init
#12 0x007fce6077b052 in SteamGameServer_InitSafe
#13 0x000000418e2838 in (wrapper managed-to-native) Steamworks.NativeMethods:SteamGameServer_Init (uint,uint16,uint16,uint16,Steamworks.EServerMode,Steamworks.InteropHelp/UTF8StringHandle)
./ServerHelper.sh: line 29:  2603 Aborted                 (core dumped) ./Unturned_Headless.x86_64 -batchmode -nographics "$@"

Running +secureserver or +InternetServer provides the same result.

This is being loaded from the original workaround lib/steamclient.so lib64/steamclient.so. However, you have to have these in order to allow multiple servers running at once or will crash.

Kept continuing to get many crashes. Even on my main servers after trying to debug everything.

Running w/o screen:

./ServerHelper.sh +InternetServer/pve1

Causes the [S_API] SteamAPI_Init(): Loaded local 'steamclient.so' OK. crash.

I'm kind of all over the place as I don't know the exact issue. Please take a look at these and maybe it will assist you, and if you do find anything. You should push all patches immedately. I don't want to wait a week of downtime for your pushes. People pay for servers and hosting, and cost them downtime because you're waiting for next week's push.

This isn't the first time this has happened and I've been here for the long haul, however, you keep breaking your game and it's pissing us, owners, off and making our players unhappy with downtime.

gegtor commented 3 years ago

I suggest a rollback to the previous stable version

mcdiggle commented 3 years ago

Can someone suggest me restart script? I just run my servers in screen with the helper script Also, I did not found any memory issues or correlation to time when the server was restarted

Interesting...I run screen too and also get crashes. Might have to move to tmux.

0 crashes and I have not restarted my servers for like 8 days

What OS? Are you using RocketMod?

Ubuntu 20.04 and yes I got rocketmod but not latest version and plugins

gegtor commented 3 years ago

What version of rocketmod or is it LDM or is it rocketmod under OpenMod

mcdiggle commented 3 years ago

What version of rocketmod or is it LDM or is it rocketmod under OpenMod

LDM ofc and version 4.9.3.3

TH3AL3X commented 3 years ago

I have ubuntu 20.04 too with the latest rocketmod, and i don't have any issue, At the beginning of the update, the script I used gave errors, and I took and started using the default one, and after that the server was working without any problem, and I have not updated my plugins for 1 month, after the update from the steam network, It is very rare that you have problems :/

TH3AL3X commented 3 years ago

I'm a bit heavy, but I'm also trying to help, you could try using this steamclient.so, this is the one I have in the linux64 folder, last modified on 22/11/2020 steamclient.so.zip Also i'm using this for start my server ./Unturned_Headless.x86_64 -batchmode -nographics +InternetServer/"$@"

gegtor commented 3 years ago

@TH3AL3X I will try running with latest steamclient.so that steam_cmd provides and see how it runs We all seem to be running deferent version

I also start my server the same way

senpai@vps:~$ md5sum steamclient.so  # TH3AL3X File
f46e5add9f0c604be48a3b257a5cad73  steamclient.so

senpai@vps:~$ md5sum Steam_CMD/linux64/steamclient.so  #steam cmd version
525aa9a18711e79977511f82a2bc776c  Steam_CMD/linux64/steamclient.so

senpai@vps:~$ md5sum /lib/steamclient.so #currently in use
fd7816ee8f14288cf63d5b8a82995a66  /lib/steamclient.so
JustOneMoreBlock commented 3 years ago

I have ubuntu 20.04 too with the latest rocketmod, and i don't have any issue, At the beginning of the update, the script I used gave errors, and I took and started using the default one, and after that the server was working without any problem, and I have not updated my plugins for 1 month, after the update from the steam network, It is very rare that you have problems :/

That's kind of the point as well. We don't update plugins often. There are only 4 main things that get updated.

At the moment, I have servers running on:

senpai@vps:~$ md5sum steamclient.so  # TH3AL3X File
f46e5add9f0c604be48a3b257a5cad73  steamclient.so

Seems to work fine, however, I was able to run all servers on the latest steam and crashed a little bit, later. So, at the moment. I don't know if this was the cause, however, I figure I should note it.