Corekeeper Dedicated server - segfault on run

[a-ndrewang]

I'm trying to run a dedicated corekeeper server on a headless OCI ARM instance. It will reproducibly segfault on attempt. I am not sure what is causing the segfault.

Environment:

OS: Ubuntu 22.04 jammy
Kernel: aarch64 Linux 6.5.0-1027-oracle
CPU: 4x ARM Neoverse-N1
RAM: 2646MiB / 23981MiB

To reproduce, I have:

• Installed box64, box86.
• Installed steamcmd.
• Installed the corekeeper dedicated server app AND the steamworks SDK Redist

  ./steamcmd.sh \
          +@sSteamCmdForcePlatformType linux \
          +force_install_dir "/home/ubuntu/corekeeper" \
          +login anonymous \
          +app_update 1963720 validate \
          +quit
  ./steamcmd.sh \
          +@sSteamCmdForcePlatformType linux \
          +force_install_dir "/home/ubuntu/corekeeper\Steamworks SDK Redist" \
          +login anonymous \
          +app_update 1007 validate \
          +quit

  I have been able to run a Valheim dedicated server instance previously with the below box64rc settings, so I have trialled the same in my .box64rc.

BOX64_DYNAREC_BLEEDING_EDGE=0
BOX64_DYNAREC_BIGBLOCK=0
BOX64_DYNAREC_STRONGMEM=2

I then try to run the server by ./_launch.sh. Of note, the launch bashfile does rely on Xvfb, as shown by this snippet:

Xvfb :99 -screen 0 1x1x24 -nolisten tcp &
xvfbpid=$!
chmod +x "$exepath" # this is the binary executable for the server
DISPLAY=:99 LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$installdir/Steamworks SDK Redist/linux64/" \
        "$exepath" -batchmode -logfile CoreKeeperServerLog.txt "$@" &

Log as below.

Dynarec for ARM64, with extension: ASIMD AES CRC32 PMULL ATOMICS SHA1 SHA2_XSERVTransSocketUNIXCreateListener: 
Will use Hardware counter measured at 25.0 MHz emulating 3.2 GHz
Params database has 80 entries
Params database has 82 entries
Box64 with Dynarec v0.3.0 62695cee built on Aug  8 2024 12:07:50
BOX64: Detected 48bits at least of address space
Counted 28 Env var
BOX64 LIB PATH: ./Steamworks SDK Redist/linux64/:./:lib/:lib64/:x86_64/:bin64/:libs64/:/lib/x86_64-linux-gnu/:/usr/lib/x86_64-linux-gnu/
BOX64 BIN PATH: ./:bin/:/usr/local/sbin/:/usr/local/bin/:/usr/sbin/:/usr/bin/:/sbin/:/bin/:/usr/games/:/usr/local/games/:/snap/bin/
Looking for ./CoreKeeperServer
Apply RC params for CoreKeeperServer
Applying BOX64_LOG=1
Applying BOX64_DYNAREC_BIGBLOCK=0
Applying BOX64_DYNAREC_STRONGMEM=1
Applying BOX64_DYNAREC_BLEEDING_EDGE=0
Applying BOX64_TRACE_FILE=/home/ubuntu/corekeeperbox64.log
BOX64 Trace redirected to "/home/ubuntu/corekeeperbox64.log"
[UnityMemory] Configuration Parameters - Can be set up in boot.config
    "memorysetup-bucket-allocator-granularity=16"
    "memorysetup-bucket-allocator-bucket-count=8"
    "memorysetup-bucket-allocator-block-size=4194304"
    "memorysetup-bucket-allocator-block-count=1"
    "memorysetup-main-allocator-block-size=16777216"
    "memorysetup-thread-allocator-block-size=16777216"
    "memorysetup-gfx-main-allocator-block-size=16777216"
    "memorysetup-gfx-thread-allocator-block-size=16777216"
    "memorysetup-cache-allocator-block-size=4194304"
    "memorysetup-typetree-allocator-block-size=2097152"
    "memorysetup-profiler-bucket-allocator-granularity=16"
    "memorysetup-profiler-bucket-allocator-bucket-count=8"
    "memorysetup-profiler-bucket-allocator-block-size=4194304"
    "memorysetup-profiler-bucket-allocator-block-count=1"
    "memorysetup-profiler-allocator-block-size=16777216"
    "memorysetup-profiler-editor-allocator-block-size=1048576"
    "memorysetup-temp-allocator-size-main=4194304"
    "memorysetup-job-temp-allocator-block-size=2097152"
    "memorysetup-job-temp-allocator-block-size-background=1048576"
    "memorysetup-job-temp-allocator-reduction-small-platforms=262144"
    "memorysetup-allocator-temp-initial-block-size-main=262144"
    "memorysetup-allocator-temp-initial-block-size-worker=262144"
    "memorysetup-temp-allocator-size-background-worker=32768"
    "memorysetup-temp-allocator-size-job-worker=262144"
    "memorysetup-temp-allocator-size-preload-manager=262144"
    "memorysetup-temp-allocator-size-nav-mesh-worker=65536"
    "memorysetup-temp-allocator-size-audio-worker=65536"
    "memorysetup-temp-allocator-size-cloud-worker=32768"
    "memorysetup-temp-allocator-size-gfx=262144"
argv[1]="-batchmode"
argv[2]="-logfile"
argv[3]="CoreKeeperServerLog.txt"
Rename process to "CoreKeeperServer"
Using emulated UnityPlayer.so
Using native(wrapped) libm.so.6
Using emulated /lib/x86_64-linux-gnu/libgcc_s.so.1
Using native(wrapped) libpthread.so.0
Using native(wrapped) libc.so.6
Using native(wrapped) ld-linux-x86-64.so.2
Using native(wrapped) libdl.so.2
Using native(wrapped) libutil.so.1
Using native(wrapped) librt.so.1
Using native(wrapped) libbsd.so.0
Using native(wrapped) libdbus-1.so.3
Using native(wrapped) libX11.so.6
Using native(wrapped) libxcb.so.1
Using native(wrapped) libXau.so.6
Using native(wrapped) libXdmcp.so.6
Using native(wrapped) libXext.so.6
Using native(wrapped) libXcursor.so.1
Using native(wrapped) libXfixes.so.3
Using native(wrapped) libXrender.so.1
Using native(wrapped) libXinerama.so.1
Using native(wrapped) libXi.so.6
Using native(wrapped) libXrandr.so.2
Using native(wrapped) libXss.so.1
Using native(wrapped) libXxf86vm.so.1
Using native(wrapped) libudev.so.1
Using emulated /home/ubuntu/corekeeper/CoreKeeperServer_Data/MonoBleedingEdge/x86_64/libmonobdwgc-2.0.so
Using native(wrapped) libz.so.1
Using emulated /home/ubuntu/corekeeper/CoreKeeperServer_Data/Plugins/discord_game_sdk.so
Using emulated /home/ubuntu/corekeeper/CoreKeeperServer_Data/Plugins/libEOSSDK-Linux-Shipping.so
Warning: Weak Symbol __gmon_start__ not found, cannot apply R_X86_64_JUMP_SLOT @0x7fff07acead0 (0x1a10786)
Using emulated /home/ubuntu/corekeeper/CoreKeeperServer_Data/Plugins/libPlayFabMultiplayer.so
Using emulated /lib/x86_64-linux-gnu/libstdc++.so.6
Warning: Weak Symbol _ITM_memcpyRtWn not found, cannot apply R_X86_64_JUMP_SLOT @0x7fff0924f060 (0x9c0f6)
Warning: Weak Symbol _ITM_RU1 not found, cannot apply R_X86_64_JUMP_SLOT @0x7fff0924f6a0 (0x9cd76)
Warning: Weak Symbol _ZGTtdlPv not found, cannot apply R_X86_64_JUMP_SLOT @0x7fff0924fb00 (0x9d636)
Warning: Weak Symbol _ITM_RU8 not found, cannot apply R_X86_64_JUMP_SLOT @0x7fff0924fff8 (0x9e026)
Warning: Weak Symbol _ITM_memcpyRnWt not found, cannot apply R_X86_64_JUMP_SLOT @0x7fff092504a8 (0x9e986)
Warning: Weak Symbol _ZGTtnam not found, cannot apply R_X86_64_JUMP_SLOT @0x7fff09250c88 (0x9f946)
Using emulated /home/ubuntu/corekeeper/CoreKeeperServer_Data/Plugins/lib_burst_generated.so
Using emulated /home/ubuntu/corekeeper/CoreKeeperServer_Data/Plugins/libparty.so
Using native(wrapped) libatomic.so.1
Using native(wrapped) libpulse.so.0
Using native(wrapped) libpulse-simple.so.0
Using emulated /home/ubuntu/corekeeper/CoreKeeperServer_Data/Plugins/libsentry.so
Using native(wrapped) libcurl.so.4
Using emulated /home/ubuntu/corekeeper/CoreKeeperServer_Data/Plugins/libsteam_api.so
Using native(wrapped) libGL.so.1
Using emulated /home/ubuntu/corekeeper/CoreKeeperServer_Data/MonoBleedingEdge/x86_64/libmono-native.so
Error loading needed lib libmonosgen-2.0.so
Warning: Cannot dlopen("libmonosgen-2.0.so"/0x4d6bb950, 102)
Using emulated /home/ubuntu/corekeeper/CoreKeeperServer_Data/MonoBleedingEdge/x86_64/libMonoPosixHelper.so
Error loading needed lib data-0x4db61330.so
Warning: Cannot dlopen("data-0x4db61330.so"/0x4db6c1b0, 101)
Error loading needed lib data-0x4dcacc40.so
Warning: Cannot dlopen("data-0x4dcacc40.so"/0x4dcbba70, 101)
Using emulated steamclient.so
Redirecting overridden malloc from symtab function for steamclient.so
Warning: Weak Symbol _ITM_RU1 not found, cannot apply R_X86_64_JUMP_SLOT @0x7fff1a277698 (0xb25700)
Warning: Weak Symbol _ZGTtnam not found, cannot apply R_X86_64_JUMP_SLOT @0x7fff1a2776a0 (0xb25700)
Warning: Weak Symbol _ITM_memcpyRtWn not found, cannot apply R_X86_64_JUMP_SLOT @0x7fff1a2776a8 (0xb25700)
Warning: Weak Symbol _ITM_RU8 not found, cannot apply R_X86_64_JUMP_SLOT @0x7fff1a2776b0 (0xb25700)
Using native(wrapped) crashhandler.so
Error loading needed lib libsteam.so
Warning: Cannot dlopen("libsteam.so"/0x7fff1503fe79, 2)
57093|Double SIGSEGV (code=1, pc=0xf2d36d6fec14, addr=0x20, prot=00)!
Sigfault/Segbus while quitting, exiting silently

[a-ndrewang]

With extra logging configs, if helpful:

BOX64_LOG=1
BOX64_ROLLING_LOG=1
BOX64_SHOWSEGV=1
BOX64_SHOWBT=1
BOX64_DLSYM_ERROR=1

174330|0x1b5844: Calling memcpy(0xE2D2E0F4CD30, 0xB1E643A0, 0x98, ...) => return 0xE2D2E0F4CD30
174330|0x7fff04170335: Calling __tls_get_addr(0x7FFF045AEEC0, 0xB1F01490, 0xB1F16240, ...) => return 0x747795A8
174330|0x7fff042406bf: Calling pthread_mutex_lock(0x7FFF045B7588, 0x600D2D80, 0xB1F16240, ...) => return 0x0
174330|0x7fff0424073a: Calling pthread_mutex_unlock(0x7FFF045B7588, 0x8, 0xFFFFFFFF, ...) => return 0x0
174330|0x7fff04256e04: Calling pthread_getspecific(0x3D, 0x0, 0xFFFFFFFF, ...) => return 0x7485C170
174330|0x7fff042403c7: Calling pthread_mutex_lock(0x7FFF045B7588, 0xE2D2E0F4CBE8, 0xFFFFFFFF, ...) => return 0x0
174330|0x7fff04240431: Calling pthread_mutex_unlock(0x7FFF045B7588, 0xE2D223F038E0, 0xFFFFFFFF, ...) => return 0x0
174330|0x7fff04256e04: Calling pthread_getspecific(0x3D, 0x0, 0xFFFFFFFF, ...) => return 0x7485C170
174330|0x7fff041123be: Calling __tls_get_addr(0x7FFF045AEEC0, 0xB1F16240, 0xB1F01490, ...) => return 0x747795A8
174330|0x7fff04170335: Calling __tls_get_addr(0x7FFF045AEEC0, 0x52B9B4, 0x7B31F680, ...) => return 0x747795A8
174330|0x7fff04170335: Calling __tls_get_addr(0x7FFF045AEEC0, 0x161169FF, 0x0, ...) => return 0x747795A8
174330|0x7fff040417ac: Calling pthread_mutex_lock(0x600D2E60, 0x7B36AAC8, 0x0, ...) => return 0x0
174330|0x7fff0404184a: Calling pthread_mutex_unlock(0x600D2E60, 0x7B36AAC8, 0xF48, ...) => return 0x0
174330|0x7fff0420339b: Calling __tls_get_addr(0x7FFF045AEEC0, 0x4DD910, 0x0, ...) => return 0x747795A8
174330|0x7fff0425b095: Calling pthread_getspecific(0x3D, 0xB1F01490, 0xB1F01490, ...) => return 0x7485C170
174330|0x7fff0425b095: Calling pthread_getspecific(0x3D, 0x2, 0x0, ...) => return 0x7485C170
NativeBT: ./CoreKeeperServer() [0x34a22684]
NativeBT: linux-vdso.so.1(__kernel_rt_sigreturn+0) [0xe2d2e12199d0]
NativeBT: ./CoreKeeperServer() [0x35020aa8]
NativeBT: ./CoreKeeperServer() [0x349f8ba8]
NativeBT: ./CoreKeeperServer(my___libc_start_main+0x250) [0x34a0e6b0]
NativeBT: ./CoreKeeperServer() [0x34a57cb0]
NativeBT: ./CoreKeeperServer() [0x34a096dc]
NativeBT: ./CoreKeeperServer() [0x3501dbcc]
NativeBT: ./CoreKeeperServer() [0x349f37ec]
NativeBT: ./CoreKeeperServer() [0x349ee1fc]
NativeBT: /lib/aarch64-linux-gnu/libc.so.6(+0x273fc) [0xe2d2e10573fc]
NativeBT: /lib/aarch64-linux-gnu/libc.so.6(__libc_start_main+0x98) [0xe2d2e10574cc]
NativeBT: ./CoreKeeperServer(_start+0x30) [0x349ee270]
EmulatedBT: /home/ubuntu/corekeeper/CoreKeeperServer_Data/Plugins/lib_burst_generated.so(Unity.Entities.EntityQueryImpl, Unity.Entities, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null.GetSingletonChunkAnd
174330|SIGSEGV @0x35020aa8 (???(./CoreKeeperServer+0x820aa8)) (x64pc=0x7fff0d9bd4f7//home/ubuntu/corekeeper/CoreKeeperServer_Data/Plugins/lib_burst_generated.so:"???", rsp=0xe2d2e0f4cb60, stack=0xe2d2e0750000:0xe2d2e0f50000 own=(nil) fp=0xe2d2e0f4cc10), for accessing 0x20 (code=1/prot=0), db=(nil)((nil):(nil)/(nil):(nil)/???:clean, hash:0/0) handler=0x7fff14030450
RAX:0x0000000000000000 RCX:0x00000000b2412e40 RDX:0x0000e2d2e0f4cc28 RBX:0x00000000b1ef1c70
RSP:0x0000e2d2e0f4cb60 RBP:0x0000e2d2e0f4cc10 RSI:0x0000e2d2e0f4cc30 RDI:0x00000000b1f69b00
 R8:0x0000e2d2e0f4cc28  R9:0x0000000000000001 R10:0x0000000000000002 R11:0x00007fff0425b07a
R12:0x0000e2d2e0f4cc2c R13:0x00000000000e0cc5 R14:0x00000000b1f69b00 R15:0x00000000b1f01490
ES:0x002b CS:0x0033 SS:0x002b DS:0x002b FS:0x0043 GS:0x0053
174330|Double SIGSEGV (code=1, pc=0x35020aa8, addr=0x20, prot=00)!
Call to dlclose(0x4)
Call to dlclose(0x5)
Call to dlclose(0x6)
Call to dlclose(0x7)
Call to dlclose(0x8)
Call to dlclose(0x9)
Call to dlclose(0xa)
Call to dlclose(0xb)
Call to dlclose(0xc)
Call to dlclose(0xd)
Call to dlclose(0xe)
Call to dlclose(0xf)
Call to dlclose(0x10)
Call to dlclose(0x11)
Call to dlclose(0x12)
Call to dlclose(0x12)
Call to dlclose(0x13)
Call to dlclose(0x14)
Call to dlclose(0x14)
Call to dlclose(0x14)
Call to dlclose(0x14)
Call to dlclose(0x15)
Call to dlclose(0x15)
Call to dlclose(0x15)
Call to dlclose(0x16)
Call to dlclose(0x17)
Call to dlclose(0x18)
Call to dlclose(0x19)
Call to dlclose(0x19)
Call to dlclose(0x1a)
Call to dlclose(0x1b)
Sigfault/Segbus while quitting, exiting silently

[Mistermaru]

I had the same problem. I don't know what box64 version you are using but I deleted my box64 folder and cloned this version with this command: "git clone --branch v0.2.4 https://github.com/ptitSeb/box64.git". Somehow it then worked for me. Running it on Raspberry Pi 4B 8GB arm64.