lloesche / valheim-server-docker

Valheim dedicated gameserver with automatic update, World backup, BepInEx and ValheimPlus mod support
https://hub.docker.com/r/lloesche/valheim-server
Apache License 2.0
1.93k stars 270 forks source link

Synology - Doesn't work after new update #401

Closed jaytark closed 3 years ago

jaytark commented 3 years ago

Hi,

Since the new updated container was released, I can no longer run the container. In fact, it sometimes hoses my server and I have to restart the server. I had to disable the ability to auto restart due to this. It was working beautifully before the recent container update, even on Hearth & Home client. I backed up the db and world files to another drive, and just deleted everything related to the docker image and reinstalled - same results.

I've even uninstalled Docker, deleted all associated files and directories, reinstalled docker, and reinstalled/configured the latest Valheim container... still no success and same error. Again, everything worked before the new container.

More Info: On Synology, I'm pulling from Docker Hub (as I've always have) and pulling the 'latest'. I'm not sure of a way to pull the old image back so we can continue to play as before. It doesn't give an option to pull a previous version.

**Update: Thumbing through real-time logs on Synology Docker, I came across this line: "valheim-updater ERROR - Failed to download Valheim server from Steam - retrying later - check your networking and volume access permissions**"

I've added firewall rule to Synology for ports 2456, 2457, although I've never needed to do that before... Problem still persists. Any help or advise would be greatly appreciated. Thanks in advance.

lloesche commented 3 years ago

What update specifically are you referring to? The base functionality of the container hasn't changed since Jun 10th. Dockerhub had stopped automatic builds for non-paying customers a while back. That should be fixed with yesterdays commit 2d2a9ad8e516a8451b5febdee62e9ab1986ea770 which builds on Github and then pushes to Dockerhub. But maybe there's a difference between the images Github and Dockerhub builds? I don't have a Synology so can't test it myself.

jaytark commented 3 years ago

Whichever is the latest image on Dockerhub. I pull "lloesche/valheim-server:latest". I had pulled it late yesterday evening after everyone got off the server from playing to update it. I only pulled it because your page said "updated xx hours ago", so I assumed you updated something to the image based on the new Valheim update. I turned off the active container, pulled lloesche/valheim-server:latest, reset the container, then started it back up. Then the error is as posted above.

kraaijmakers commented 3 years ago

I got the same problem, it sometimes freezes the whole Synology NAS.

slindebe commented 3 years ago

The latest Valheim patch today changed something. IMHO it has nothing to do with the image.

I had HTOP running in a SSH window when my Synology became unresponsive, and I could see that a steamcmd process was taking all available CPU resources.

image

I couldn't create another SSH session, and all other containers and apps were unresponsive.

It happened after the Valheim server was updated to the latest patch today.

jaytark commented 3 years ago

@slindebe Last night, my server was running it fine with my friends playing on it... I hadn't updated the docker image in some time (probably in a month or two) As soon as they got off last night, I updated the docker image and hasn't been able to run the game since... and I tested it immediately after I refreshed the container with the new image.... not saying it's the image, but not saying it isn't either... maybe just coincidence... Not sure what else to do at this point.

I haven't even fired up the Valheim client now on PC because I know my server isn't running it correctly and inaccessible based on the logs... and the RAM utilization only stays at around 56MB for the container... as before, it was well over 1Gig... the CPU utilization stays at around 40-56%, as before, around 15-20% when playing the game and when it was running correctly. It's as though it's caught in a loop and it freezes my server at times by possibly hammering the CPU.

All of my other container apps run fine.

slindebe commented 3 years ago

I haven't updated the container for a long time, but the way it's set up it updates the Valheim server from Steam regularly. So my container hasn't changed but the Valheim patch through Steam changed something.

jaytark commented 3 years ago

For the heck of it, I tried a different Valheim Docker Image through the Dockerhub. (It was the second most popular one. Lloesche's being the first most popular one based on star-ratings). After trying a different one, it works fine. I can get in and play. It's just Lloesche's image that's not working for me. Which I hate, because Lloesche's has better options with Discord Webhooks and such. If Lloesche's works for me at some point, I'll switch back to his.

Note: This isn't to disrespect Lloesche at all, I'm just desperately trying to troubleshoot this the best way I know how. I appreciate Lloesche's work and efforts.

tomekduda commented 3 years ago

Same here, also on Synology. "Failed to download Valheim server from Steam - retrying later - check your networking and volume access permissions" in the log. with all permissions I to the folders and with entries in syno firewall. I've even opened terminal with the container running and basic stuff like ping works fine so no idea how it could be network issue.

Left it running for a while and it was a mistake, 2 hard reboots today. The mbround18/valheim seems to download the server fine, shame about the features I've grown to rely on

nicjay commented 3 years ago

Experiencing the same issue here on Synology DS918+. I tried to start a fresh new container today using the included stock docker-compose, changing only my custom server name and password environment vars. Both the Docker daemon and Synology DSM itself hung after doing a docker-compose up -d, ultimately forcing a hard reboot of the device.

The mbround18/valheim image previously mentioned does work for the time being.

jaytark commented 3 years ago

I even did a manual pull of lloesche's github image version through ssh and manually re-entered the environmental variables with a fresh install and it still hangs.

lloesche commented 3 years ago

Can anyone post their entire container logs please? From startup to when the error occurs? I don't have access to a Synology but I noticed that on one of our servers valheim-updater fails with the following message

Sep 19 07:35:08 supervisord: valheim-updater Connecting anonymously to Steam Public...
Sep 19 07:35:10 supervisord: valheim-updater OK
Sep 19 07:35:10 supervisord: valheim-updater Waiting for client config...OK
Sep 19 07:35:10 supervisord: valheim-updater Waiting for user info...
Sep 19 07:35:16 supervisord: valheim-updater /opt/steamcmd/steamcmd.sh: line 38:   889 Killed                  $DEBUGGER "$STEAMEXE" "$@"
Sep 19 07:35:16 supervisord: valheim-updater ERROR - Failed to update Valheim server from Steam - however an existing version was found locally - using it

And on another machine the step Waiting for user info... takes incredibly long but then succeeds.

When I manually run Steam to update valheim I get this

root@18229210b028:/opt/steamcmd# /opt/steamcmd/steamcmd.sh +login anonymous +force_install_dir /opt/valheim/dl/server +app_update 896660 validate +quit
Redirecting stderr to '/root/Steam/logs/stderr.txt'
[  0%] Checking for available updates...
[----] Verifying installation...
Steam Console Client (c) Valve Corporation
-- type 'quit' to exit --
Loading Steam API...OK

Connecting anonymously to Steam Public...OK
Waiting for client config...OK
Waiting for user info...OK
 Update state (0x3) reconfiguring, progress: 0.00 (0 / 0)
 Update state (0x5) verifying install, progress: 16.17 (171258717 / 1059006974)
 Update state (0x5) verifying install, progress: 56.73 (600796844 / 1059006974)
 Update state (0x5) verifying install, progress: 97.98 (1037628119 / 1059006974)
Success! App '896660' fully installed.
Terminating m_ThreadClient, likely to crash down the line... but avoiding hang on exit
/opt/steamcmd/steamcmd.sh: line 38:   967 Killed                  $DEBUGGER "$STEAMEXE" "$@"

Anyone else seeing this too?

lloesche commented 3 years ago

On the effected system I'm able to reproduce the issue just by entering and existing steam, not even attempting to download any update:

root@18229210b028:/opt/steamcmd# ./steamcmd.sh
Redirecting stderr to '/root/Steam/logs/stderr.txt'
Looks like steam didn't shutdown cleanly, scheduling immediate update check
[  0%] Checking for available updates...
[----] Verifying installation...
Steam Console Client (c) Valve Corporation
-- type 'quit' to exit --
Loading Steam API...OK

Steam>quit
Work thread 'CJobMgr::m_WorkThreadPool:1' is marked exited, but we could not immediately join prior to deleting -- proceeding without join
Terminating m_ThreadClient, likely to crash down the line... but avoiding hang on exit
./steamcmd.sh: line 38:  1712 Killed                  $DEBUGGER "$STEAMEXE" "$@"

Doing the same on another system works just fine:

root@513e0c7d8b04:/opt/steamcmd# ./steamcmd.sh
Redirecting stderr to '/root/Steam/logs/stderr.txt'
[  0%] Checking for available updates...
[----] Verifying installation...
Steam Console Client (c) Valve Corporation
-- type 'quit' to exit --
Loading Steam API...OK

Steam>quit
root@513e0c7d8b04:/opt/steamcmd#

Leading me to believe that there is currently a bug in Steam depending on the Kernel version it is running on. I would assume that Synology NAS appliances are rather conservative with their Kernel updates and don't update too often which might result in the same behavior? This is all just speculation however. I would need for several people to post their container startup logs and also their Kernel versions would be interesting. I'll actually add some debug output to the container so we'll have that automatically in the future.

I also opened https://github.com/ValveSoftware/steam-for-linux/issues/8083 to see if maybe this is a known issue.

tomekduda commented 3 years ago

Run from scratch (deleted the container, image, pulled fresh from dockerhub. I'd need to ssh over to pull from ghcr)

Basic config (run without elevated privileges, basic mounting, not readonly...) ![image](https://user-images.githubusercontent.com/7957690/133917712-982f5dc2-ea7e-4d32-998b-284df9596039.png)

On start CPU usage briefly jumped to 22%, now hovers under 1%. 81 MB RAM, from what I remember on my 1 year old config it was ~20+ and 1-2 GB on idle

Log: lloesche-valheim-server.csv

I let it run for few min and it became quite unresponsive (CPU and RAM didn't jump, the Syno UI still worked but I was unable to stop the container normally, start a bash terminal inside the container... SSH asked for password and also hung). Force kill resulted in msg similar to "Docker's API interface crashed".

This post helps? https://old.reddit.com/r/synology/comments/cn9qnd/what_distribution_of_linux_is_synology_using/

From SSH (I'm an "admin" according to Synology UI but I'm not sitting on root account) I don't see mention of Debian anywhere. "toster" is the NAS's name so "Linux toster" won't mean anything to anybody.

eyescream@toster:~$ uname -a
Linux toster 3.10.105 #25556 SMP Sat Aug 28 02:13:34 CST 2021 x86_64 GNU/Linux synology_braswell_916+

eyescream@toster:~$ cat /etc/VERSION
majorversion="6"
minorversion="2"
major="6"
minor="2"
micro="4"
productversion="6.2.4"
buildphase="GM"
buildnumber="25556"
smallfixnumber="2"
nano="0"
base="25556"
builddate="2021/08/28"
buildtime="14:40:29"

I started the container for few min, took a snapshot and stopped it. I can then explore the snapshot's filesystem without impacting the container / letting it freeze the system again.

docker ps
docker commit d395d3d6293f  mysnapshot
docker run -t -i mysnapshot /bin/bash

The Steam/logs/stderr.txt:

root@d716c7c91549:/# cat /home/valheim/Steam/logs/stderr.txt
src/tier0/threadtools.cpp (4071) : Assertion Failed: Probably deadlock or failure waiting for thread to initialize.
crash_20210919074534_5.dmp[78]: Uploading dump (out-of-process)
/tmp/dumps/crash_20210919074534_5.dmp
CWorkThreadPool::StartWorkThread: Thread creation failed.

crash_20210919074534_5.dmp[78]: Finished uploading minidump (out-of-process): success = yes
crash_20210919074534_5.dmp[78]: response: Discarded=1
crash_20210919074534_5.dmp[78]: file ''/tmp/dumps/crash_20210919074534_5.dmp'', upload yes: ''Discarded=1''

The dump file is weird part binary thing and my Linux skills end here, sorry. How do I get the file from Windows 10 cmd -> ssh -> sudo su -> container snapshot exploration back to Windows... I'll paste what I can here, if you actually need the binary form I'll need some tips :)

/tmp/dumps/crash_20210919074534_5.dmp ``` root@d716c7c91549:/# cat /tmp/dumps/crash_20210919062456_5.dmp MDMP FaD8xgG0gGgG|gGgG@ gG gGH `?P``R@@@0U0A_0l0l����l l~8l8~,��0 o,P, [BP0?0 PTl;Ԁ ȃ,?Ёȃ< Pn,|w0J P >,P 8vu�� Vu/P vs u����v,p+u@Рl ` @РʠD'"w 1631572845)c :'i'u]ʠ8?Š @K̊����|̊[, u l4 [[.[P[5g'u��n8 ].9`Wuz;6 ].K>Ϡ suTsQh C6QŠ5����Іu`Assert( Assertion Failed: Probably deadlock or failure waiting for thread to initialize. ):/data/src/tier0/threadtools.cpp:4071 Yˉ؉n)����u`vr `v u`vv`vX[v OO����OO���` ������ (L erPid: 0uX[vl3 W;0TracerPid: 0 ing) &``wXuMX;[pu0K ̊J̊LJL̊l(MvLuu`v`vu`vvqvv`v`vyuv>GL^  H ����uЍ٨uLЍ@PL[܍o܍AL[$^iCuLЍ@^n|wAssertion Failed: Probably deadlock or failure waiting for thread to in L1ҹ։S̀[=vމ0 $61҅du8dbJ0>bg>b0>bJ|KJj[8LJ0>b70>b7j[8L0>b0>b0>b0>bg>bg>b0>bg>b|vopP`Nb1[`@nWpxKo0>b< (L����(LopKh u@NHL'\U[ u@N����V(Lס[����LP%Zס[ס[u@N����L@NvLP[pv|xuvpvmuNd����PMNPMN@M |MbNbH,0 `V< `Bp b%, b����,όp b,0 `_,NWp b_ 8< ` crash_20210919062456_2 ] l]Kl] l]����K]G,]:: ] ]K]]|]\~@_,~8]0 bD^4 \\K^X^|^q[, <^4 q[k[`ڷ[j[4q[_ K_G,_:[:i@B K__|_[[ڷ[,\`m?a_ 4 [[Y[8aY[ [[4a[_[36 l]`0l]_`_l] ^l]~8_l]__~8___ ]2056] bb b]^]\l_] b b\l_< b b b b# bb bb + 6 b ` b | b v b| c__)bu`66v buu�������� b6D^` b b6n_,̑`u����P ����vav۷[̑,``��̑,��rc,��i`aiu����aP����u����u`����av$@bvxG8v[[[[[$b$[[[[[[[[[[[$bx b 1631572845`ulPR@bJr p0b _@{ c bbbp.b{PPRpv? 1#g+z?5L{ilinux-gate.solinux-gate.soP`;`Q@@0U_0?J<`RLGenuineIntelzLinux 3.10.105 #25556 SMP Sat Aug 28 02:13:34 CST 2021 x86_64processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 76 model name : Intel(R) Pentium(R) CPU N3710 @ 1.60GHz stepping : 4 microcode : 0x404 cpu MHz : 1601.000 cache size : 1024 KB physical id : 0 siblings : 4 core id : 0 cpu cores : 4 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 movbe popcnt tsc_deadline_timer aes rdrand lahf_lm 3dnowprefetch ida arat epb invpcid_single tpr_shadow vnmi flexpriority ept vpid tsc_adjust smep erms bogomips : 3199.85 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 76 model name : Intel(R) Pentium(R) CPU N3710 @ 1.60GHz stepping : 4 microcode : 0x404 cpu MHz : 1601.000 cache size : 1024 KB physical id : 0 siblings : 4 core id : 1 cpu cores : 4 apicid : 2 initial apicid : 2 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 movbe popcnt tsc_deadline_timer aes rdrand lahf_lm 3dnowprefetch ida arat epb invpcid_single tpr_shadow vnmi flexpriority ept vpid tsc_adjust smep erms bogomips : 3199.85 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: processor : 2 vendor_id : GenuineIntel cpu family : 6 model : 76 model name : Intel(R) Pentium(R) CPU N3710 @ 1.60GHz stepping : 4 microcode : 0x404 cpu MHz : 1601.000 cache size : 1024 KB physical id : 0 siblings : 4 core id : 2 cpu cores : 4 apicid : 4 initial apicid : 4 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 movbe popcnt tsc_deadline_timer aes rdrand lahf_lm 3dnowprefetch ida arat epb invpcid_single tpr_shadow vnmi flexpriority ept vpid tsc_adjust smep erms bogomips : 3199.85 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: processor : 3 vendor_id : GenuineIntel cpu family : 6 model : 76 model name : Intel(R) Pentium(R) CPU N3710 @ 1.60GHz stepping : 4 microcode : 0x404 cpu MHz : 1601.000 cache size : 1024 KB physical id : 0 siblings : 4 core id : 3 cpu cores : 4 apicid : 6 initial apicid : 6 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 movbe popcnt tsc_deadline_timer aes rdrand lahf_lm 3dnowprefetch ida arat epb invpcid_single tpr_shadow vnmi flexpriority ept vpid tsc_adjust smep erms bogomips : 3199.85 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: Name: steamcmd State: t (tracing stop) Tgid: 63 Pid: 63 PPid: 59 TracerPid: 66 Uid: 0 0 0 0 Gid: 0 0 0 0 FDSize: 256 Groups: VmPeak: 21824 kB VmSize: 21012 kB VmLck: 0 kB VmPin: 0 kB VmHWM: 5132 kB VmRSS: 5132 kB VmData: 5648 kB VmStk: 132 kB VmExe: 6280 kB VmLib: 3168 kB VmPTE: 56 kB VmSwap: 0 kB Threads: 3 SigQ: 2/31418 SigPnd: 0000000000000000 ShdPnd: 0000000000000000 SigBlk: 0000000000010000 SigIgn: 0000000001000000 SigCgt: 00000001800014e8 CapInh: 00000000a80425fb CapPrm: 00000000a80425fb CapEff: 00000000a80425fb CapBnd: 00000000a80425fb Cpus_allowed: f Cpus_allowed_list: 0-3 Mems_allowed: 1 Mems_allowed_list: 0 voluntary_ctxt_switches: 29 nonvoluntary_ctxt_switches: 27 /opt/steamcmd/linux32/steamcmd+loginanonymous+force_install_dir/opt/valheim/dl/server+app_update896660validate+quitSERVER_NAME=eyescreamSUPERVISOR_GROUP_NAME=valheim-updaterVALHEIM_LOG_FILTER_STARTSWITH=(Filename:SUPERVISOR_SERVER_URL=unix:///var/run/supervisor.sockHOSTNAME=lloesche-valheim-serverSERVER_PASS=higithubVALHEIM_LOG_FILTER_MATCH= PWD=/opt/steamcmdTZ=Etc/UTCVALHEIM_LOG_FILTER_UTF8=trueHOME=/home/valheimLANG=en_US.UTF-8TERM=xtermUSER=valheimSHLVL=2VALHEIM_LOG_FILTER_EMPTY=trueLD_LIBRARY_PATH=/opt/steamcmd/linux32:LC_CTYPE=C.UTF-8SUPERVISOR_PROCESS_NAME=valheim-updaterWORLD_NAME=HiGithubPATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/binVALHEIM_LOG_FILTER_STARTSWITH_AssertionFailed=src/steamnetworkingsockets/clientlib/steamnetworkingsockets_lowlevel.cppDEBIAN_FRONTEND=noninteractiveOLDPWD=/SUPERVISOR_ENABLED=1_=/opt/steamcmd/linux32/steamcmd 0s!sd4p  Ϟf6300000-f6321000 rw-p 00000000 00:00 0 f6321000-f6400000 ---p 00000000 00:00 0 f64ff000-f6500000 ---p 00000000 00:00 0 f6500000-f6600000 rw-p 00000000 00:00 0 [stack:26000] f6600000-f66a3000 rw-p 00000000 00:00 0 f66a3000-f6700000 ---p 00000000 00:00 0 f6767000-f6768000 ---p 00000000 00:00 0 f6768000-f6868000 rw-p 00000000 00:00 0 [stack:25999] f6868000-f6935000 r-xp 00000000 00:1c 6119 /opt/steamcmd/linux32/crashhandler.so f6935000-f6939000 r--p 000cc000 00:1c 6119 /opt/steamcmd/linux32/crashhandler.so f6939000-f693a000 rw-p 000d0000 00:1c 6119 /opt/steamcmd/linux32/crashhandler.so f693a000-f6946000 rw-p 00000000 00:00 0 f6946000-f6bc0000 r--p 0006d000 00:1c 8223 /usr/lib/locale/locale-archive f6bc0000-f6dc0000 r--p 00000000 00:1c 8223 /usr/lib/locale/locale-archive f6dc0000-f6dc3000 rw-p 00000000 00:00 0 f6dc3000-f6de0000 r--p 00000000 00:1c 6017 /lib/i386-linux-gnu/libc-2.31.so f6de0000-f6f35000 r-xp 0001d000 00:1c 6017 /lib/i386-linux-gnu/libc-2.31.so f6f35000-f6fa5000 r--p 00172000 00:1c 6017 /lib/i386-linux-gnu/libc-2.31.so f6fa5000-f6fa6000 ---p 001e2000 00:1c 6017 /lib/i386-linux-gnu/libc-2.31.so f6fa6000-f6fa8000 r--p 001e2000 00:1c 6017 /lib/i386-linux-gnu/libc-2.31.so f6fa8000-f6faa000 rw-p 001e4000 00:1c 6017 /lib/i386-linux-gnu/libc-2.31.so f6faa000-f6fac000 rw-p 00000000 00:00 0 f6fac000-f6fb8000 r--p 00000000 00:1c 6036 /lib/i386-linux-gnu/libm-2.31.so f6fb8000-f7079000 r-xp 0000c000 00:1c 6036 /lib/i386-linux-gnu/libm-2.31.so f7079000-f70ae000 r--p 000cd000 00:1c 6036 /lib/i386-linux-gnu/libm-2.31.so f70ae000-f70af000 r--p 00101000 00:1c 6036 /lib/i386-linux-gnu/libm-2.31.so f70af000-f70b0000 rw-p 00102000 00:1c 6036 /lib/i386-linux-gnu/libm-2.31.so f70b0000-f70b2000 r--p 00000000 00:1c 6054 /lib/i386-linux-gnu/librt-2.31.so f70b2000-f70b7000 r-xp 00002000 00:1c 6054 /lib/i386-linux-gnu/librt-2.31.so f70b7000-f70b9000 r--p 00007000 00:1c 6054 /lib/i386-linux-gnu/librt-2.31.so f70b9000-f70ba000 r--p 00008000 00:1c 6054 /lib/i386-linux-gnu/librt-2.31.so f70ba000-f70bb000 rw-p 00009000 00:1c 6054 /lib/i386-linux-gnu/librt-2.31.so f70bb000-f70c1000 r--p 00000000 00:1c 6050 /lib/i386-linux-gnu/libpthread-2.31.so f70c1000-f70d2000 r-xp 00006000 00:1c 6050 /lib/i386-linux-gnu/libpthread-2.31.so f70d2000-f70d9000 r--p 00017000 00:1c 6050 /lib/i386-linux-gnu/libpthread-2.31.so f70d9000-f70da000 r--p 0001d000 00:1c 6050 /lib/i386-linux-gnu/libpthread-2.31.so f70da000-f70db000 rw-p 0001e000 00:1c 6050 /lib/i386-linux-gnu/libpthread-2.31.so f70db000-f70dd000 rw-p 00000000 00:00 0 f70dd000-f70de000 r--p 00000000 00:1c 6025 /lib/i386-linux-gnu/libdl-2.31.so f70de000-f70e0000 r-xp 00001000 00:1c 6025 /lib/i386-linux-gnu/libdl-2.31.so f70e0000-f70e1000 r--p 00003000 00:1c 6025 /lib/i386-linux-gnu/libdl-2.31.so f70e1000-f70e2000 r--p 00003000 00:1c 6025 /lib/i386-linux-gnu/libdl-2.31.so f70e2000-f70e3000 rw-p 00004000 00:1c 6025 /lib/i386-linux-gnu/libdl-2.31.so f70e5000-f70eb000 rw-p 00000000 00:00 0 f70eb000-f70ec000 r--p 00000000 00:1c 6010 /lib/i386-linux-gnu/ld-2.31.so f70ec000-f7109000 r-xp 00001000 00:1c 6010 /lib/i386-linux-gnu/ld-2.31.so f7109000-f7114000 r--p 0001e000 00:1c 6010 /lib/i386-linux-gnu/ld-2.31.so f7115000-f7116000 r--p 00029000 00:1c 6010 /lib/i386-linux-gnu/ld-2.31.so f7116000-f7117000 rw-p 0002a000 00:1c 6010 /lib/i386-linux-gnu/ld-2.31.so f7117000-f7739000 r-xp 00000000 00:1c 6124 /opt/steamcmd/linux32/steamcmd f7739000-f773a000 r-xp 00000000 00:00 0 [vdso] f773a000-f775e000 r--p 00622000 00:1c 6124 /opt/steamcmd/linux32/steamcmd f775e000-f7765000 rw-p 00646000 00:1c 6124 /opt/steamcmd/linux32/steamcmd f7765000-f778d000 rw-p 00000000 00:00 0 f8910000-f8a50000 rw-p 00000000 00:00 0 [heap] ff9cc000-ff9ed000 rw-p 00000000 00:00 0 [stack] О Huss 0l}<_dlinux-gate.so.1
kraaijmakers commented 3 years ago

Message me if you need more info, I am willing to help but I rather not crash the whole Synology Unit again, I can even let you debug it since my set-up has been redone since yesterday and the only thing I started running was your Valheim server before I stopped.

$ uname -a Linux DSM01 4.4.180+ #41890 SMP Thu Jul 15 03:43:42 CST 2021 x86_64 GNU/Linux synology_apollolake_218+

$ cat /etc/VERSION majorversion="7" minorversion="0" major="7" minor="0" micro="0" productversion="7.0" buildphase="GM" buildnumber="41890" smallfixnumber="0" nano="0" base="41890"

$ docker -v Docker version 20.10.3, build b455053

lloesche commented 3 years ago

@tomekduda @kraaijmakers thank you, I have updated the issue at Valve Software with the information you provided. The issue is very likely a combination of Kernel version plus libc or sdl libraries used in the Docker container with a recent change in steamcmd. I just had a thought related to this. This Docker container is using debian:stable-slim as base image. Debian just released version 11 (Bullseye) which superseeds 10 (Buster) in August. Even though there were no changes to the container itself other than README updates in the past couple of months, since the container is using the debian:stable-slim tag even a README update would have caused a rebuild against the new version of Debian. Maybe steamcmd is not compatible with the latest version of Debian stable. Or rather with the libraries used in it. Maybe it's even a combination of new Debian, old Kernel or something like that causing the issue. I will see if I can downgrade to an older version of Debian and if that helps with the issue.

lloesche commented 3 years ago

Ok, I have downgraded the Docker container from Debian stable 11.0 to 10.0. Can you see if that makes any difference?

JHaney commented 3 years ago

FIXED! Great work time to get hunting again! For the record I nuked the container and image effectively starting from scratch but pointing at my old game data dirs.

image

tomekduda commented 3 years ago

Looks like it's solved. I also had to nuke, download, configure from scratch. Upgrade in place (as described in readme) didn't help, but that might be Synology caching playing tricks.

The CPU and memory usage are "more like it", 25-30% and 1.7-2.0 GB on idle.

Log (don't ask me why it's sorted by time descending): lloesche-valheim-server1.csv

I'll leave it running for few min, see if it freezes the system.

lloesche commented 3 years ago

Sounds good! Would be great if @jaytark who originally reported the bug could validate if it is fixed in their system as well before we close the issue.

I also updated the issue at Valve with our findings. Should be easy for them to reproduce.

kraaijmakers commented 3 years ago

Are you guys it running through ghcr.io or Docker Hub because the ghcr.io image still freezes my Synology NAS. Gonna try the Docker hub now.

jaytark commented 3 years ago

I'll be able to try the new fix later this evening and will report the results on this thread.

schutzmuster commented 3 years ago

I only created an account to say thank you for the fast fix. My Synology DS920+ freezed and now works as before. I recreated the container from scratch, too. But the saved world works, so nothing missing. I used the Image from ghcr.io.

Server running in idle now for 9 hours. Steam Update every hour.

kraaijmakers commented 3 years ago

Hmm seems to work for now, when I did the ghcr.io image from the command line it somehow didn't redownload it I think. Via the UI of Synology it did. Thanks guys for the fix. and @lloesche thanks for the support on the container.

lloesche commented 3 years ago

@tomekduda I was asked in the Valve issue if it's possible for you to provide the dmp binary file. They would need that for further debugging.

tomekduda commented 3 years ago

Bugger. I'm happy to help but I deleted the container's snapshot in the meantime and now of course I'm on your latest version. Can you make a broken build with different tag, do-not-use, debug or similar name? I could try forking your repo but I've never built anything on Docker

lloesche commented 3 years ago

@tomekduda oh sure, Github containe registry keeps all versions:

docker pull ghcr.io/lloesche/valheim-server@sha256:2cc61cb267192d34c73526e7377a6cc7c3eefd843916af81047305ada14721d2

this is the build from 3 days ago. It will have the broken behaviour.

jaytark commented 3 years ago

I'm pleased to confirm the current change fixed the issue for me. I'm able to enter the server's instance now. Will try and report of any issues with saves, etc as friends join and play, but I suspect there won't be any. Thanks @lloesche for the support and fine work on this matter and others who supplied supporting information.

grease10 commented 3 years ago

This wasn't just docker images I don't believe. I was trying this image on Saturday (for the first time ever) running on a CentOS7 machine and I had the same issue. I manually downloaded steamcmd in the container and it would result in Assertion Failed: Probably deadlock or failure waiting for thread to initialize error when I tried to run steamcmd alone. I could run the same outside the container on centos without issue.

I just tried the new container and it starts up with no issue.

iloveicedgreentea commented 3 years ago

I can confirm the issue before this fix on kubernetes on GKE. Using the latest docker image did not fix the valheim-updater issue. Will try to wipe the config and try again.

Edit: wiping out the persistent volume claim for /opt/valheim fixed the issue for me.

slindebe commented 3 years ago

I can confirm that the image with downgraded Debian works fine for me ina Synology environment.

Thanks @lloesche!

jaytark commented 3 years ago

Had a friend try last night, and it still works well. So far, it's doing good.

lloesche commented 3 years ago

Thanks everyone for testing and confirming the downgrade to Debian 10 helped. I'm keeping an eye on the Valve thread but seems this is a general issue with Debian 11 libc and older Linux kernels.

ddulic commented 1 year ago

https://github.com/lloesche/valheim-server-docker/pull/508 broke the image for Synology.

Had to use valheim-server@sha256:17ba123cddda6af6408e407d24c9e0150db82534feae9fea3720bfe7e3aeff5a

javijuji commented 1 year ago

Good afternoon. Trying to do a fresh install and getting full system freeze on Synology DSM 7.1 similar to what was mentioned last year.

slindebe commented 1 year ago

Good afternoon. Trying to do a fresh install and getting full system freeze on Synology DSM 7.1 similar to what was mentioned last year.

Use lloesche/valheim-server:debian10 instead, kernel/docker version problem as far as I understand.

Pizzaboy140 commented 1 year ago

After making a valheim container using debian10, it runs, CPU hovers around 12%, and RAM at ~3.5GB, so I assume it's running as expected. However, when I try connecting from the Valheim Client, it spins at "Connecting" for a while, before throwing the error "Failed to connect".

I don't think its a networking issue because if I run a Valheim server on my PC itself (forwarding to my PC instead of Synology NAS, hosting the container) I'm able to get into my Valheim World.

Would anyone know what the cause could be?

tomekduda commented 1 year ago

@Pizzaboy140 try getting there from Steam's list of servers. It didn't want to work for me from in-game connector or a desktop shortcut ("C:\Program Files (x86)\Steam\steam.exe" -applaunch 892970 +connect IP.GOES.HE.RE:2456 +password hunter2) but after I tried from Steam client's view-> servers something clicked.

Pizzaboy140 commented 1 year ago

Hmm, when I try that, it claims "Server is not responding". I am a docker newbie, but it seems odd that it seems like the world keeps getting rebuilt.

In case it means anything to anyone, please see the following for html of my container's logs: valheim-server.txt *html converted to .txt

It looks like the Server does "connect" in the logs, but there's probably something else going wrong that I can't figure out...

Pizzaboy140 commented 1 year ago

welp, figured it out. I had a firewall up on my NAS itself. I was letting in TCP ports 2456-2458.

However, Valheim uses UDP.

Once I made a UDP port rule, I could connect. Hence why the game attempted to connect, but got stopped...

FYI in case this saves others hours of troubleshooting hehe.

TheUntouchable commented 1 year ago

I just tried to set up a new Valheim server on my Synology NAS (218+) and had some problems downloading Valheim to the NAS. It started and after a few MB download it stopped.

Found this thread here and tried the debian10 image, unfortunately I still have the same problem, just a little bit later: grafik

Anything I am doing wrong here? ;)

sudo docker run -d --name valheim-server --cap-add=sys_nice --stop-timeout 120 -p 2456-2457:2456-2457/udp -v /volume1/docker/valheim/config:/config -v /volume1/docker/valheim/data:/opt/valheim -e TZ="Europe/Berlin" -e SERVER_NAME="" -e WORLD_NAME="" -e SERVER_PASS="" -e VALHEIM_PLUS=true --restart=always lloesche/valheim-server:debian10

TheUntouchable commented 1 year ago

Ok, just restarted the container and this time it worked.. Nevermind :D