steamcmd deadlock and complete system freeze under Synology DSM 7.1 after latest bump to debian bullseye

[NanoWireDnB]

With the latest pull #508 the valheim server container has the same issues as in #401 Also:

https://github.com/ValveSoftware/steam-for-linux/issues/8083

Deadlock of the steamcmd thread and a full freeze of the Synology NAS.

2022-10-14 13:47:31,280 INFO spawned: 'valheim-updater' with pid 45 Oct 14 13:47:31 supervisord: valheim-updater DEBUG - [45] - Running Valheim Server updater as user valheim uid 0 Oct 14 13:47:31 supervisord: valheim-updater INFO - Found existing PID file - checking process Oct 14 13:47:31 supervisord: valheim-updater DEBUG - [45] - Sending SIGUSR1 to PID 51 Oct 14 13:47:31 supervisord: valheim-updater INFO - Removing stale PID file and starting run Oct 14 13:47:31 supervisord: valheim-updater INFO - Releasing PID file /var/run/valheim/valheim-updater.pid Oct 14 13:47:31 supervisord: valheim-updater DEBUG - [45] - Kernel: Linux Valheim-Server 4.4.180+ #42962 SMP Tue Sep 20 22:45:13 CST 2022 x86_64 GNU/Linux Oct 14 13:47:31 supervisord: valheim-updater DEBUG - [45] - Found CPU with 2100.000 MHz Oct 14 13:47:31 supervisord: valheim-updater DEBUG - [45] - Memory total/free/available: 16349856/11130008/11844952 Oct 14 13:47:31 supervisord: valheim-updater DEBUG - [45] - Storage configuration: Oct 14 13:47:31 supervisord: valheim-updater Filesystem Size Used Avail Use% Mounted on Oct 14 13:47:31 supervisord: valheim-updater /dev/mapper/cachedev_0 21T 13T 8.7T 59% / Oct 14 13:47:31 supervisord: valheim-updater /dev/mapper/cachedev_0 21T 13T 8.7T 59% /config Oct 14 13:47:31 supervisord: valheim-updater /dev/mapper/cachedev_0 / btrfs rw,nodev,relatime,ssd,synoacl,space_cache=v2,auto_reclaim_space,metadata_ratio=50,syno_allocator,subvolid=74708,subvol=/@syno/@docker/btrfs/subvolumes/b3ef82e1845006e54b3c66e6ed105b3e6e1e95370a5af70542564d6c9c958e1b 0 0 Oct 14 13:47:31 supervisord: valheim-updater /dev/mapper/cachedev_0 /config btrfs rw,nodev,relatime,ssd,synoacl,space_cache=v2,auto_reclaim_space,metadata_ratio=50,syno_allocator,subvolid=47341,subvol=/@syno/docker/valheim/config 0 0 Oct 14 13:47:31 supervisord: valheim-updater /dev/mapper/cachedev_0 /opt/valheim btrfs rw,nodev,relatime,ssd,synoacl,space_cache=v2,auto_reclaim_space,metadata_ratio=50,syno_allocator,subvolid=47341,subvol=/@syno/docker/valheim/data 0 0 Oct 14 13:47:31 supervisord: valheim-updater DEBUG - [45] - Valheim updater was just started - skipping connected players check Oct 14 13:47:31 supervisord: valheim-updater INFO - Downloading/updating/validating Valheim server from Steam Oct 14 13:47:31 supervisord: valheim-updater Redirecting stderr to '/home/valheim/Steam/logs/stderr.txt' > Oct 14 13:47:31 supervisord: valheim-updater Looks like steam didn't shutdown cleanly, scheduling immediate update check Oct 14 13:47:31 assert_20221014134731_5.dmp[88]: Uploading dump (out-of-process) /tmp/dumps/assert_20221014134731_5.dmp 2022-10-14 13:47:31,834 INFO success: syslogd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-10-14 13:47:31,834 INFO success: crond entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2022-10-14 13:47:32,688 INFO success: valheim-updater entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) > Oct 14 13:47:32 supervisord: valheim-updater src/tier0/threadtools.cpp (4122) : Probably deadlock or failure waiting for thread to initialize. Oct 14 13:47:32 supervisord: valheim-updater [ 0%] Checking for available updates... Oct 14 13:47:32 supervisord: valheim-updater Thread failed to initialize > Oct 14 13:47:32 supervisord: valheim-updater src/tier0/threadtools.cpp (4122) : Probably deadlock or failure waiting for thread to initialize. Oct 14 13:47:32 supervisord: valheim-updater Thread failed to initialize Oct 14 13:47:32 supervisord: valheim-bootstrap valheim-updater: started

Oct 14 13:49:11 supervisord: valheim-updater Terminated Oct 14 13:49:11 supervisord: valheim-updater ERROR - Failed to update Valheim server from Steam - however an existing version was found locally - using it Oct 14 13:49:23 assert_20221014134731_5.dmp[88]: Finished uploading minidump (out-of-process): success = yes Oct 14 13:49:23 assert_20221014134731_5.dmp[88]: response: Discarded=1 Oct 14 13:49:23 assert_20221014134731_5.dmp[88]: file ''/tmp/dumps/assert_20221014134731_5.dmp'', upload yes: ''Discarded=1''

I'm amazed that this is even possible for an unprivileged process, but to be honest, Synology is know for using old kernels.

Their newest OS version DSM 7.1.1 uses Linux kernel version 4.4.180. (yes, the one from 2016).

Disclaimer: I used to work for Synology, so I can say their Kernel is heavily patched and not prone to security issues, but as we see opens up another chapter of unforeseen compatibility issues. And in the foreseeable future, the procedure (using an old, albeit stable kernel and providing it with the latest security patches and some backports of its own) will unfortunately not change. Their environment is too heavily modified for that. I don't like it, I think it's the wrong way. (and there you have one of the reasons why I don't work there anymore).

I am fully aware, that downgrading is not an option, as Debian Bullseye with kernel 5.10_LTS and newer c and c++ lib version are needed for playfab and the new Valheim crossplay feature.

As mentioned in #504 it would be nice if you can create a new branch/tag for NAS which stays on Debian 10.

JK, thank you in advance for considering it and your awesome work on this Docker Container @lloesche :-)

[Senectus]

sigh same problem here.

[Concretv]

Hi, same issue on my side. I was able tu make it runing until yesterday (by desactivating steam update and server restart) but the server needs to be updated after new Valheim patch (yesterday) and this causes Synology NAS to hang (DS720+ on DSM7.1).

[Senectus]

I tried the other Valheim docker image with the same effect :-/

[00BlackRose00]

same issue here :(

[Mansonjr]

Same issue unable to put the server working. :(

[esteis]

I'm having the same issue after the latest valheim patch

[lloesche]

There is now a new Docker image lloesche/valheim-server:debian10 which uses the older Debian 10 base image, but therefor doesn't support crossplay. This should be compatible with any NAS using old Kernels.

[00BlackRose00]

There is now a new Docker image lloesche/valheim-server:debian10 which uses the older Debian 10 base image, but therefor doesn't support crossplay. This should be compatible with any NAS using old Kernels.

It's working again! Thank you so much :)

[LeventeHudak]

Debian10 branch was working perfectly fine in the last weeks and then suddenly yesterday(12.12) it started throwing issues with libparty.so, when server boots up. Now we can't connect to the server anymore. Attached the logs for reference. _lloesche-valheim-server1_logs (1).txt

fresh start: lloesche-valheim-server1.csv

[opello]

After #544 it should be worth trying :latest again.