build issue "./CoreKeeperServer: No such file or director"

[defeek]

Using Docker Compose added corekeeper to the volume to house in a single folder in my docker folder

  #corekeeper
  core-keeper:
    container_name: corekeeper
    image: arguser/core-keeper-dedicated
    volumes:
      - ./corekeeper/server-files:/home/steam/core-keeper-dedicated
      - ./corekeeper/server-data:/home/steam/core-keeper-data
    env_file:
      - ./envs/corekeeper.env
    restart: always

When I am trying to build, I am getting this error in the logs

chmod: cannot access './CoreKeeperServer': No such file or directory
Started server process with pid 122
./launch.sh: line 38: ./CoreKeeperServer: No such file or directory
_XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.

I don't know what I am overlooking here but would appreciate any help

[arguser]

I can't replicate your issue, please double check that the folders exist and verify you have writing access.

Here's my docker-compose.yml

version: "3"

services: 
  core-keeper:
      container_name: corekeeper
      image: arguser/core-keeper-dedicated:latest
      volumes:
        - ./corekeeper/server-files:/home/steam/core-keeper-dedicated
        - ./corekeeper/server-data:/home/steam/core-keeper-data
      env_file:
        - ./envs/corekeeper.env
      restart: always

Here is my folder tree, bear in mind that I've created the folders corekeeper, server-files and server-data before running.

.
├── corekeeper
│   ├── server-data
│   │   ├── ServerConfig.json
│   │   └── ServerConfig.json.pugbackup
│   └── server-files
│       ├── CoreKeeperServer
│       ├── CoreKeeperServer_Data
│       │   ├── Plugins
│       │   │   ├── lib_burst_generated.so
│       │   │   └── libsteam_api.so
│       │   ├── Resources
│       │   │   ├── UnityPlayer.png
│       │   │   └── unity default resources
│       │   ├── RuntimeInitializeOnLoads.json
│       │   ├── ScriptingAssemblies.json
│       │   ├── app.info
│       │   ├── boot.config
│       │   ├── data.unity3d
│       │   ├── extra-types.txt
│       │   └── il2cpp_data
│       │       ├── Metadata
│       │       │   └── global-metadata.dat
│       │       ├── Resources
│       │       │   └── mscorlib.dll-resources.dat
│       │       └── etc
│       │           └── mono
│       │               ├── 2.0
│       │               │   ├── Browsers
│       │               │   │   └── Compat.browser
│       │               │   ├── DefaultWsdlHelpGenerator.aspx
│       │               │   ├── machine.config
│       │               │   ├── settings.map
│       │               │   └── web.config
│       │               ├── 4.0
│       │               │   ├── Browsers
│       │               │   │   └── Compat.browser
│       │               │   ├── DefaultWsdlHelpGenerator.aspx
│       │               │   ├── machine.config
│       │               │   ├── settings.map
│       │               │   └── web.config
│       │               ├── 4.5
│       │               │   ├── Browsers
│       │               │   │   └── Compat.browser
│       │               │   ├── DefaultWsdlHelpGenerator.aspx
│       │               │   ├── machine.config
│       │               │   ├── settings.map
│       │               │   └── web.config
│       │               ├── browscap.ini
│       │               ├── config
│       │               └── mconfig
│       │                   └── config.xml
│       ├── GameAssembly.so
│       ├── GameID.txt
│       ├── README.txt
│       ├── UnityPlayer.so
│       ├── _launch.sh
│       ├── _readme.sh
│       ├── launch.sh
│       ├── linux64
│       │   └── steamclient.so
│       ├── steamapps
│       │   ├── appmanifest_1007.acf
│       │   ├── appmanifest_1963720.acf
│       │   ├── downloading
│       │   └── temp
│       └── steamclient.so
├── docker-compose.yml
└── envs
    └── corekeeper.env

[defeek]

When i docker exec -it corekeeper /bin/bash i cannot find any /CoreKeeperServer/ folder anywhere is the problem i believe since it errors during launch.sh

feek@fsrv:~/docker/corekeeper$ docker exec -it corekeeper /bin/bash
steam@5e1b157294f5:~$ ls
Steam  core-keeper-data  core-keeper-dedicated  entry.sh  launch.sh  steamcmd
steam@5e1b157294f5:~$ cd core-keeper-dedicated/
steam@5e1b157294f5:~/core-keeper-dedicated$ ls -a
.  ..

Also no files or folders ever get pulled into the server-data/ or server-files/ folders locally

feek@fsrv:~/docker$ tree ~/docker/corekeeper/
/home/feek/docker/corekeeper/
├── server-data
└── server-files

full install logs

feek@fsrv:~/docker$ docker logs -f corekeeper
Redirecting stderr to '/home/steam/Steam/logs/stderr.txt'
[  0%] Checking for available updates...
[----] Downloading update (0 of 19,198 KB)...
[  0%] Downloading update (2,349 of 19,198 KB)...
[ 12%] Downloading update (4,719 of 19,198 KB)...
[ 24%] Downloading update (6,088 of 19,198 KB)...
[ 31%] Downloading update (7,353 of 19,198 KB)...
[ 38%] Downloading update (8,597 of 19,198 KB)...
[ 44%] Downloading update (9,819 of 19,198 KB)...
[ 51%] Downloading update (11,062 of 19,198 KB)...
[ 57%] Downloading update (12,423 of 19,198 KB)...
[ 64%] Downloading update (13,667 of 19,198 KB)...
[ 71%] Downloading update (14,912 of 19,198 KB)...
[ 77%] Downloading update (16,102 of 19,198 KB)...
[ 83%] Downloading update (17,471 of 19,198 KB)...
[ 91%] Downloading update (18,634 of 19,198 KB)...
[ 97%] Downloading update (19,198 of 19,198 KB)...
[100%] Download complete.
[----] Installing update...
[----] Extracting package...
[----] Extracting package...
[----] Extracting package...
[----] Extracting package...
[----] Installing update...
[----] Installing update...
[----] Installing update...
[----] Installing update...
[----] Installing update...
[----] Installing update...
[----] Installing update...
[----] Installing update...
[----] Cleaning up...
[----] Update complete, launching Steamcmd...
Redirecting stderr to '/home/steam/Steam/logs/stderr.txt'
[  0%] Checking for available updates...
[----] Verifying installation...
Steam Console Client (c) Valve Corporation - version 1654574676
-- type 'quit' to exit --
Loading Steam API...OK

Connecting anonymously to Steam Public...OK
Waiting for client config...OK
Waiting for user info...OK
 Update state (0x3) reconfiguring, progress: 0.00 (0 / 0)
 Update state (0x61) downloading, progress: 6.29 (4194304 / 66706004)
Success! App '1007' fully installed.
 Update state (0x3) reconfiguring, progress: 0.00 (0 / 0)
 Update state (0x61) downloading, progress: 4.77 (13631488 / 286032978)
 Update state (0x61) downloading, progress: 83.97 (240169876 / 286032978)
Success! App '1963720' fully installed.
Redirecting stderr to '/home/steam/Steam/logs/stderr.txt'
[  0%] Checking for available updates...
[----] Verifying installation...
Steam Console Client (c) Valve Corporation - version 1654574676
-- type 'quit' to exit --
Loading Steam API...OK

Connecting anonymously to Steam Public...OK
Waiting for client config...OK
Waiting for user info...OK
Success! App '1007' already up to date.
Success! App '1963720' already up to date.
chmod: cannot access './CoreKeeperServer': No such file or directory
Started server process with pid 122
./launch.sh: line 38: ./CoreKeeperServer: No such file or directory
_XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.

[arguser]

• Are you using the latest docker image version?
• Also, check if you don't have any old docker image for cm2network/steamcmd.
• On the other hand check the folders owners with ls -lha.

[defeek]

Latest docker image i do have valheim and vrising servers but theyre in their own containers so shouldnt mess with this one.

feek@fsrv:~/docker$ docker images
REPOSITORY                        TAG       IMAGE ID       CREATED        SIZE
arguser/core-keeper-dedicated     latest    528b09dad8e2   45 hours ago   830MB
linuxserver/plex                  latest    bb415ee5b671   2 days ago     651MB
linuxserver/freshrss              latest    2f1a05ef5cc4   2 days ago     120MB
tautulli/tautulli                 latest    c93deb79d8da   3 weeks ago    199MB
trueosiris/vrising                latest    3a1d62f40b60   3 weeks ago    2.32GB
ghcr.io/xnaas/caddy               stable    9f90625c117e   7 weeks ago    53.5MB
lscr.io/linuxserver/thelounge     latest    68dfbbebf0bd   2 months ago   251MB
sctx/overseerr                    latest    a3cd181a21cf   2 months ago   571MB
containrrr/watchtower             latest    333de6ea525a   5 months ago   16.9MB
ghcr.io/lloesche/valheim-server   latest    0de9badeed6e   8 months ago   519MB

1026 is my uid

feek@fsrv:~/docker$ docker exec -it corekeeper /bin/bash
steam@da7d25a59250:~$ ls -lha
total 80K
drwxr-xr-x 1 steam steam 4.0K Jun 25 14:39 .
drwxr-xr-x 1 root  root  4.0K May 14 12:47 ..
-rw------- 1 steam steam  361 Jun 25 14:42 .bash_history
-rw-r--r-- 1 steam steam  220 Aug  4  2021 .bash_logout
-rw-r--r-- 1 steam steam 3.5K Aug  4  2021 .bashrc
drwxr-xr-x 3 steam steam 4.0K Jun 25 11:57 .cache
drwxr-xr-x 3 steam steam 4.0K Jun 25 14:39 .config
drwxr-xr-x 3 steam steam 4.0K Jun 25 14:19 .local
-rw-r--r-- 1 steam steam  807 Aug  4  2021 .profile
drwxr-xr-x 1 steam steam 4.0K Jun 25 14:41 .steam
drwxr-xr-x 1 steam steam 4.0K Jun 25 14:41 Steam
drwxrwxr-x 2  1026 users 4.0K Jun 25 00:37 core-keeper-data
drwxrwxr-x 2  1026 users 4.0K Jun 25 00:37 core-keeper-dedicated
-rwxr-xr-x 1 steam steam  736 Jun 23 23:48 entry.sh
-rwxr-xr-x 1 steam steam 1.1K Jun 23 23:48 launch.sh
drwxr-xr-x 1 steam steam 4.0K Jun 25 14:41 steamcmd

[arguser]

You could try to create a file from inside the container in the folders core-keeper-data or core-keeper-dedicated. I guess you'll see you don't have permission. Check envs to see if they are properly set or try to run launch.sh again.

[defeek]

I changed owner of those two folders to steam:steam

feek@fsrv:~/docker$ docker exec -it -u 0 corekeeper chown steam:steam -R /home/steam/core-keeper-dedicated/
feek@fsrv:~/docker$ docker exec -it -u 0 corekeeper chown steam:steam -R /home/steam/core-keeper-data/

This is my log output now no longer getting error about that file missing but also cannot connect to the server with that GameID where is folder /tmp/ supposed to be? or is that /tmp/ outside of the steam home dir in the container?

Connecting anonymously to Steam Public...OK
Waiting for client config...OK
Waiting for user info...OK
Success! App '1007' already up to date.
Success! App '1963720' already up to date.
Redirecting stderr to '/home/steam/Steam/logs/stderr.txt'
/tmp/dumps is not owned by us - delete and recreate
Unable to delete /tmp/dumps. Continuing anyway.
[  0%] Checking for available updates...
[----] Verifying installation...
Steam Console Client (c) Valve Corporation - version 1654574676
-- type 'quit' to exit --
Loading Steam API...OK

Connecting anonymously to Steam Public...OK
Waiting for client config...OK
Waiting for user info...OK
Success! App '1007' already up to date.
Success! App '1963720' already up to date.
Started server process with pid 58
_XSERVTransmkdir: Owner of /tmp/.X11-unix should be set to root
Game ID: AQPbjsguaVZj6e8LMlFPk15qnWpH

tree output

feek@fsrv:~/docker/corekeeper$ tree
.
├── server-data
│   ├── ServerConfig.json
│   └── ServerConfig.json.pugbackup
└── server-files
    ├── CoreKeeperServer
    ├── CoreKeeperServer_Data
    │   ├── app.info
    │   ├── boot.config
    │   ├── data.unity3d
    │   ├── extra-types.txt
    │   ├── il2cpp_data
    │   │   ├── etc
    │   │   │   └── mono
    │   │   │       ├── 2.0
    │   │   │       │   ├── Browsers
    │   │   │       │   │   └── Compat.browser
    │   │   │       │   ├── DefaultWsdlHelpGenerator.aspx
    │   │   │       │   ├── machine.config
    │   │   │       │   ├── settings.map
    │   │   │       │   └── web.config
    │   │   │       ├── 4.0
    │   │   │       │   ├── Browsers
    │   │   │       │   │   └── Compat.browser
    │   │   │       │   ├── DefaultWsdlHelpGenerator.aspx
    │   │   │       │   ├── machine.config
    │   │   │       │   ├── settings.map
    │   │   │       │   └── web.config
    │   │   │       ├── 4.5
    │   │   │       │   ├── Browsers
    │   │   │       │   │   └── Compat.browser
    │   │   │       │   ├── DefaultWsdlHelpGenerator.aspx
    │   │   │       │   ├── machine.config
    │   │   │       │   ├── settings.map
    │   │   │       │   └── web.config
    │   │   │       ├── browscap.ini
    │   │   │       ├── config
    │   │   │       └── mconfig
    │   │   │           └── config.xml
    │   │   ├── Metadata
    │   │   │   └── global-metadata.dat
    │   │   └── Resources
    │   │       └── mscorlib.dll-resources.dat
    │   ├── Plugins
    │   │   ├── lib_burst_generated.so
    │   │   └── libsteam_api.so
    │   ├── Resources
    │   │   ├── unity default resources
    │   │   └── UnityPlayer.png
    │   ├── RuntimeInitializeOnLoads.json
    │   └── ScriptingAssemblies.json
    ├── GameAssembly.so
    ├── GameID.txt
    ├── _launch.sh
    ├── launch.sh
    ├── linux64
    │   └── steamclient.so
    ├── _readme.sh
    ├── README.txt
    ├── steamapps
    │   ├── appmanifest_1007.acf
    │   ├── appmanifest_1963720.acf
    │   ├── downloading
    │   └── temp
    ├── steamclient.so
    └── UnityPlayer.so

21 directories, 44 files

[defeek]

so update It seems to be working but i cannot connect to my own server my friends can connect thank you for the help i will close this now :)

[arguser]

Great to know that you solved it @defeek, could you describe what was going for future reference?

[defeek]

it started working after i changed owner of those two folders in the container to steam:steam i wonder if there is a way to check/do that before the build?

feek@fsrv:~/docker$ docker exec -it -u 0 corekeeper chown steam:steam -R /home/steam/core-keeper-dedicated/
feek@fsrv:~/docker$ docker exec -it -u 0 corekeeper chown steam:steam -R /home/steam/core-keeper-data/

[iqbalmineraltown]

in my case, the game ID is generated after I did these in the following order:

1. run container // the output says as written on the issue description
2. chown core-keeper folders to steam user as @defeek mentioned
3. re-run entry.sh // i tried to run launch.sh first, but the output is still the same as first
4. re-run launch.sh

[xcjs]

I wouldn't consider this closed - the container should be updated to set permissions on those directories properly.

I've also run into the same issue. This will also be an issue any time the container is updated through my CI/CD process or created in general.

[Frooodle]

I can confirm I had to do the same chown

[arguser]

in my case, the game ID is generated after I did these in the following order:

1. **run container**  // the output says as written on the issue description

2. **chown core-keeper folders to steam user as @defeek mentioned**

3. **re-run entry.sh** // i tried to run launch.sh first, but the output is still the same as first

4. **re-run launch.sh**

Have you tried to pre-create the folders? I'm finding that docker-compose create the volumes (folders in this case) as root, which leads to this issue.

Created by running docker-compose without previously creating the folders leads to the issue in my case:

drwxr-xr-x 2 root    root    4.0K Jul 11 18:49 server-data
drwxr-xr-x 2 root    root    4.0K Jul 11 18:49 server-files
---
core-keeper_1  | ./launch.sh: line 38: ./CoreKeeperServer: No such file or directory
core-keeper_1  | _XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.

[iqbalmineraltown]

I see. I can confirm this, running docker without pre-creating the folder will lead to those folders owned by root

Yes, pre-creating folders before running docker will generate gameID.txt without any issue!

I thought I didn't need to run pre-create steps if I'm using docker-compose. This was totally my mistake when reading the README

Have you tried to pre-create the folders? I'm finding that docker-compose create the volumes (folders in this case) as root, which leads to this issue.

Created by running docker-compose without previously creating the folders leads to the issue in my case:

drwxr-xr-x 2 root    root    4.0K Jul 11 18:49 server-data
drwxr-xr-x 2 root    root    4.0K Jul 11 18:49 server-files
---
core-keeper_1  | ./launch.sh: line 38: ./CoreKeeperServer: No such file or directory
core-keeper_1  | _XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.

[arguser]

I wouldn't consider this closed - the container should be updated to set permissions on those directories properly.

I've also run into the same issue. This will also be an issue any time the container is updated through my CI/CD process or created in general.

Please refer to this comment https://github.com/escapingnetwork/core-keeper-dedicated/issues/5#issuecomment-1180904078

[xcjs]

That's not a proper solution. At the very least, you should expose UID and GID (user ID/group ID) environment variables so the process of creating the container can be fully deterministic and create the server user based on those values.

The permissions used inside the container can clash with existing user accounts on the system it's deployed on, and in my case it does.

You can also instead affect ownership of the mounted volumes from inside the container ahead of time.

[arguser]

@xcjs thanks for pointing that out, I'll try to get an alternative with the environment variables. If you have something cooked up already a PR is more than welcome :)

[xcjs]

@xcjs thanks for pointing that out, I'll try to get an alternative with the environment variables. If you have something cooked up already a PR is more than welcome :)

I'll look into submitting something in the next few weeks when I have time away from other projects to look at it.

I think there was another issue regarding the X11 /tmp mount that prevented Azeos' Wilderness and Sunken Sea from generating past a certain point that I fixed for my container, but that's more of a documentation fix.

I don't mean to be overly-critical - this project definitely saved a lot of time, and I thank you for that.

[Frooodle]

@xcjs thanks for pointing that out, I'll try to get an alternative with the environment variables. If you have something cooked up already a PR is more than welcome :)

I'll look into submitting something in the next few weeks when I have time away from other projects to look at it.

I think there was another issue regarding the X11 /tmp mount that prevented Azeos' Wilderness and Sunken Sea from generating past a certain point that I fixed for my container, but that's more of a documentation fix.

I don't mean to be overly-critical - this project definitely saved a lot of time, and I thank you for that.

Can you expand on what you meant by the tmp mounting or give a example so others can do that fix too?

[xcjs]

Can you expand on what you meant by the tmp mounting or give a example so others can do that fix too?

Sure! Core Keeper eventually switches to GPU rendering for generating farther away areas. In order to do this, it needs access to the X11 socket accessible inside /tmp/.X11-unix to access the host's X11 socket and therefore the graphics hardware. The container/server log actually warns about that particular path not being accessible otherwise.

In my docker-compose, I expose this as a volume:

version: "3"

services:
  core-keeper:
    container_name: core-keeper
    image: escaping/core-keeper-dedicated:latest
    volumes:
      - REDACTED:/home/steam/core-keeper-dedicated
      - REDACTED:/home/steam/core-keeper-data
      - /tmp/.X11-unix:/tmp/.X11-unix
    environment:
      - WORLD_INDEX=0
      - WORLD_NAME=REDACTED
      - WORLD_SEED=0
      - GAME_ID=REDACTED
      - DATA_PATH=/home/steam/core-keeper-data
      - MAX_PLAYERS=8
    restart: unless-stopped
    stop_grace_period: 2m

[Frooodle]

Can you expand on what you meant by the tmp mounting or give a example so others can do that fix too?

Sure! Core Keeper eventually switches to GPU rendering for generating farther away areas. In order to do this, it needs access to the X11 socket accessible inside /tmp/.X11-unix to access the host's X11 socket and therefore the graphics hardware. The container/server log actually warns about that particular path not being accessible otherwise.

In my docker-compose, I expose this as a volume:

version: "3"

services:
  core-keeper:
    container_name: core-keeper
    image: escaping/core-keeper-dedicated:latest
    volumes:
      - REDACTED:/home/steam/core-keeper-dedicated
      - REDACTED:/home/steam/core-keeper-data
      - /tmp/.X11-unix:/tmp/.X11-unix
    environment:
      - WORLD_INDEX=0
      - WORLD_NAME=REDACTED
      - WORLD_SEED=0
      - GAME_ID=REDACTED
      - DATA_PATH=/home/steam/core-keeper-data
      - MAX_PLAYERS=8
    restart: unless-stopped
    stop_grace_period: 2m

Interesting, never knew this, does this effectively mean you must have a GPU to run this game long term as a requirement? Looks like getting x11 on my unraid server requires a VM or something rather than native.. damm

Edit: guess this x11 graphics stuff should move to different issue #

[arguser]

Edit: guess this x11 graphics stuff should move to different issue #

Done

[xcjs]

Can you expand on what you meant by the tmp mounting or give a example so others can do that fix too?

Sure! Core Keeper eventually switches to GPU rendering for generating farther away areas. In order to do this, it needs access to the X11 socket accessible inside /tmp/.X11-unix to access the host's X11 socket and therefore the graphics hardware. The container/server log actually warns about that particular path not being accessible otherwise.

In my docker-compose, I expose this as a volume:

version: "3"

services:
  core-keeper:
    container_name: core-keeper
    image: escaping/core-keeper-dedicated:latest
    volumes:
      - REDACTED:/home/steam/core-keeper-dedicated
      - REDACTED:/home/steam/core-keeper-data
      - /tmp/.X11-unix:/tmp/.X11-unix
    environment:
      - WORLD_INDEX=0
      - WORLD_NAME=REDACTED
      - WORLD_SEED=0
      - GAME_ID=REDACTED
      - DATA_PATH=/home/steam/core-keeper-data
      - MAX_PLAYERS=8
    restart: unless-stopped
    stop_grace_period: 2m

Interesting, never knew this, does this effectively mean you must have a GPU to run this game long term as a requirement? Looks like getting x11 on my unraid server requires a VM or something rather than native.. damm

Edit: guess this x11 graphics stuff should move to different issue #

Sorry, I thought I already replied to this. The server does require a graphics backend to run the server out past a certain distance in the Wilderness/Sunken Sea areas - eventually you'll run into unbreakable wall sections where you shouldn't.

(Of note are some unbreakable wall sections that act as placeholders for yet to be released biomes - those are expected.)

While graphics hardware is a huge help to this end, if you have absolutely no graphics hardware otherwise, I think you can use xvfb on the host as I mentioned in #7.

[nmlynch94]

I also was having this problem and tracked down that it was a permissions issue (common problem when using bind-mounts). I just swapped to using Docker Volumes for server files and data instead and that's working fine since Docker can manage those file permissions. The files can still be accessed if needed and it bypasses the bind-mounting headaches. Any reason not to recommend that by default? Anyone who knows enough about Docker to know they want bind mounts instead will have the knowledge to solve those permissions issues and change the compose-file and it will be an easier setup for less-experienced users. I'm happy to make a PR to update the README if that sounds good.

Thanks for making this! Saved me a good chunk of time.

version: "3"

services:
  core-keeper:
    image: escaping/core-keeper-dedicated
    volumes:
      - server-files:/home/steam/core-keeper-dedicated
      - server-data:/home/steam/core-keeper-data
      - /tmp/.X11-unix:/tmp/.X11-unix
    environment:
      - WORLD_INDEX=0
      - WORLD_NAME=NACHOLE
      - WORLD_SEED=0
      - GAME_ID=UkelslKJDF34899dJFLSJke
      - DATA_PATH=
      - MAX_PLAYERS=10
    restart: always
    stop_grace_period: 2m
volumes:
    server-files:
    server-data:

[arguser]

Hi @nmlynch94 you can go ahead with the PR, thank you!

[nmlynch94]

No problem. Done https://github.com/escapingnetwork/core-keeper-dedicated/pull/11/files.

[arguser]

Merged!

[xcjs]

No problem. Done https://github.com/escapingnetwork/core-keeper-dedicated/pull/11/files.

This is definitely a great way to handle this. I prefer host mounts to volumes due to some of my backup software, but this is definitely an improvement.