steamcmd missing file permissions

dichromatic commented 5 months ago

Describe the bug

Palworld unable to update on container restart, steamcmd gives me a missing file permissions error. Have tried recreating the container to the same result. Not exactly sure what's causing the permissions error, I haven't touched the container other than restarting it for game updates. Have not touched PUID or GUID either.

To Reproduce

Steps to reproduce the behavior:

Restart container with docker-compose up
Wait for steamcmd to update Palworld

Expected behavior

Palworld should have updated.

Actual behavior

On startup, it gives me "ERROR! Failed to install app '2394010' (Missing file permissions)".

Screenshots

N/A

OS information (IMPORTANT)

OS: Debian 11 bullseye
Palworld Version: 0.2.1.0
Docker Image Version: latest 0.37.1

Hardware information (IMPORTANT)

CPU: Intel i3-12100F
Memory: 32gb DDR4

docker-compose.yml contents (IMPORTANT)

docker-compose.yml

```yaml version: '3' services: palworld: image: thijsvanloef/palworld-server-docker:latest restart: unless-stopped container_name: palworld-server ports: - 10000:10000/udp - 27015:27015/udp environment: #- ADMIN_PASSWORD="pword" - SERVER_NAME="name" - PORT=10000 - PLAYERS=16 - MULTITHREADING=true - UPDATE_ON_BOOT=true - COMMUNITY=false # Enable this if you want your server to show up in the community servers tab, USE WITH SERVER_PASSWORD! # Enable the environment variables below if you have COMMUNITY=true # - SERVER_PASSWORD="worldofpals" # - SERVER_NAME="World of Pals" # - ADMIN_PASSWORD="someAdminPassword" volumes: - ./palworld:/palworld/ ```

Container/Host Logs (IMPORTANT)

Logs

```log palworld-server | Redirecting stderr to '/root/Steam/logs/stderr.txt' palworld-server | Logging directory: '/root/Steam/logs' palworld-server | [ 0%] Checking for available updates... palworld-server | [----] Verifying installation... palworld-server | Steam Console Client (c) Valve Corporation - version 1718305764 palworld-server | -- type 'quit' to exit -- palworld-server | Loading Steam API...OK palworld-server | palworld-server | Connecting anonymously to Steam Public...OK palworld-server | Waiting for client config...OK palworld-server | Waiting for user info...OK palworld-server | ERROR! Failed to install app '2394010' (Missing file permissions) palworld-server | ./PalServer.sh -port=10000 -players=16 -servername="name" -useperfthreads -NoAsyncLoadingThread -UseMultithreadForDS palworld-server | *****STARTING SERVER*****The file already exists: /palworld/Pal/Binaries/Linux/steamclient.so palworld-server | [S_API] SteamAPI_Init(): Loaded local 'steamclient.so' OK. palworld-server | CAppInfoCacheReadFromDiskThread took 0 milliseconds to initialize palworld-server | Setting breakpad minidump AppID = 2394010 palworld-server | [S_API FAIL] Tried to access Steam interface SteamUser021 before SteamAPI_Init succeeded. palworld-server | [S_API FAIL] Tried to access Steam interface SteamFriends017 before SteamAPI_Init succeeded. palworld-server | [S_API FAIL] Tried to access Steam interface STEAMAPPS_INTERFACE_VERSION008 before SteamAPI_Init succeeded. palworld-server | [S_API FAIL] Tried to access Steam interface SteamNetworkingUtils004 before SteamAPI_Init succeeded. ```

Additional context

Running on a home debian server.

Bug Report Checklist

[X] I searched for my issue on the Known issues page
[X] I searched for my issue in the Closed issues
[X] I have included all relevant information required above.
[X] I am willing to fix this bug

nevesm commented 5 months ago

I am facing the same issue with appId '730'

thijsvanloef commented 5 months ago

@dichromatic Can you show the permissions on your mounted folder?

ls -al ./palworld

dichromatic commented 4 months ago

❯ ls -al palworld
total 44648
drwxr-xr-x 7 rumi rumi     4096 Jun 16 20:09 .
drwxr-xr-x 3 rumi rumi     4096 Jan 24 16:32 ..
-rwxr-xr-x 1 rumi rumi     2087 Apr  5 22:45 DefaultPalWorldSettings.ini
-rwxr--r-- 1 rumi rumi     4096 Jan 24 16:32 ._.DS_Store
-rwxr--r-- 1 rumi rumi     8196 Jan 24 16:33 .DS_Store
drwxr-xr-x 6 rumi rumi     4096 Apr  5 22:47 Engine
drwxr-xr-x 2 rumi rumi     4096 Jan 21 16:22 linux64
-rwxr-xr-x 1 rumi rumi      438 Apr  5 22:45 Manifest_DebugFiles_Linux.txt
-rwxr-xr-x 1 rumi rumi     3031 Apr  5 22:45 Manifest_NonUFSFiles_Linux.txt
-rwxr-xr-x 1 rumi rumi  7990849 Apr  5 22:45 Manifest_UFSFiles_Linux.txt
drwxr-xr-x 7 rumi rumi     4096 Jan 24 16:33 Pal
-rwxr-xr-x 1 rumi rumi      815 Apr  5 22:45 PalServer.sh
drwxr-xr-x 3 rumi rumi     4096 Jun 16 20:11 steamapps
drwxr-xr-x 4 rumi rumi     4096 Apr  7 04:03 steamapps_backup
-rwxr-xr-x 1 rumi rumi 37665020 Jan 21 16:18 steamclient.so

rumi being my user. I tried recreating the steamapps folder, the original one is labeled under steamapps_backup

thijsvanloef commented 4 months ago

You might want to try the following command:

id

Should get an output like this:

uid=1000(thijs) gid=1000(thijs)

And then run your container with:

    environment:
      PUID: 1000 (your UID)
      PGID: 1000 (your GID)

thijsvanloef commented 4 months ago

Hi @dichromatic are you still experiencing this issue?

dichromatic commented 4 months ago

Sorry was a little busy, I tried changing the PUID and GUID in the docker-compose file and it still gives me the same error.

thijsvanloef commented 4 months ago

If you try to set everything in the folder to 777 does it work?

sudo chmod -R 777 ./palworld

This is to figure out if this is a permission or a different issue, this is kind of unsafe to run the server like this.

ecterun commented 4 months ago

I ran into the same issue described in the issue. I ran the container as the user that the files were owned by, and this allowed the container to download the game files correctly. The problem then was I ran into an error were it was unable to actually start the game server successfully.

In the container logs it would ask for a password and show access denied as if the container entered in a bad pass.

I was able to then re-run the container as the original user and the game server started because the files were successfully downloaded and put in the correct location.

ecterun commented 4 months ago

This is the logs with the using the user id/gid that owns the files. It installs the game update correctly, but is unable to start the server sucessfully. To do that I need to rebuild the container without the --user settings.

*****STARTING INSTALL/UPDATE*****tid(13) burning pthread_key_t == 0 so we never use it
Redirecting stderr to '/home/steam/Steam/logs/stderr.txt'
Logging directory: '/home/steam/Steam/logs'
[  0%] Checking for available updates...
[----] Downloading update (0 of 58,905 KB)...
[  0%] Downloading update (927 of 58,905 KB)...
[  1%] Downloading update (3,901 of 58,905 KB)...
[  6%] Downloading update (6,538 of 58,905 KB)...
[ 11%] Downloading update (8,450 of 58,905 KB)...
[ 14%] Downloading update (10,232 of 58,905 KB)...
[ 17%] Downloading update (11,886 of 58,905 KB)...
[ 20%] Downloading update (13,516 of 58,905 KB)...
[ 22%] Downloading update (15,505 of 58,905 KB)...
[ 26%] Downloading update (17,127 of 58,905 KB)...
[ 29%] Downloading update (18,706 of 58,905 KB)...
[ 31%] Downloading update (20,535 of 58,905 KB)...
[ 34%] Downloading update (22,249 of 58,905 KB)...
[ 37%] Downloading update (23,927 of 58,905 KB)...
[ 40%] Downloading update (25,685 of 58,905 KB)...
[ 43%] Downloading update (27,176 of 58,905 KB)...
[ 46%] Downloading update (29,148 of 58,905 KB)...
[ 49%] Downloading update (30,536 of 58,905 KB)...
[ 51%] Downloading update (32,429 of 58,905 KB)...
[ 55%] Downloading update (34,059 of 58,905 KB)...
[ 57%] Downloading update (35,797 of 58,905 KB)...
[ 60%] Downloading update (37,491 of 58,905 KB)...
[ 63%] Downloading update (39,058 of 58,905 KB)...
[ 66%] Downloading update (40,511 of 58,905 KB)...
[ 68%] Downloading update (41,247 of 58,905 KB)...
[ 70%] Downloading update (42,261 of 58,905 KB)...
[ 71%] Downloading update (43,096 of 58,905 KB)...
[ 73%] Downloading update (43,951 of 58,905 KB)...
[ 74%] Downloading update (44,822 of 58,905 KB)...
[ 76%] Downloading update (45,776 of 58,905 KB)...
[ 77%] Downloading update (46,655 of 58,905 KB)...
[ 79%] Downloading update (47,506 of 58,905 KB)...
[ 80%] Downloading update (48,377 of 58,905 KB)...
[ 82%] Downloading update (49,331 of 58,905 KB)...
[ 83%] Downloading update (50,234 of 58,905 KB)...
[ 85%] Downloading update (51,228 of 58,905 KB)...
[ 86%] Downloading update (52,143 of 58,905 KB)...
[ 88%] Downloading update (52,990 of 58,905 KB)...
[ 89%] Downloading update (53,813 of 58,905 KB)...
[ 91%] Downloading update (54,799 of 58,905 KB)...
[ 93%] Downloading update (55,761 of 58,905 KB)...
[ 94%] Downloading update (56,593 of 58,905 KB)...
[ 96%] Downloading update (57,603 of 58,905 KB)...
[ 97%] Downloading update (58,521 of 58,905 KB)...
[ 99%] Downloading update (58,905 of 58,905 KB)...
[100%] Download complete.
[----] Installing update...
[----] Extracting package...
[----] Extracting package...
[----] Extracting package...
[----] Extracting package...
[----] Installing update...
[----] Installing update...
[----] Installing update...
[----] Installing update...
[----] Installing update...
[----] Installing update...
[----] Installing update...
[----] Installing update...
[----] Cleaning up...
[----] Update complete, launching Steamcmd...
Redirecting stderr to '/home/steam/Steam/logs/stderr.txt'
Logging directory: '/home/steam/Steam/logs'
[  0%] Checking for available updates...
[----] Verifying installation...
UpdateUI: skip show logoSteam Console Client (c) Valve Corporation - version 1718904263
-- type 'quit' to exit --
Loading Steam API...OK

Connecting anonymously to Steam Public...OK
Waiting for client config...OK
Waiting for user info...OK
 Update state (0x3) reconfiguring, progress: 0.00 (0 / 0)
 Update state (0x3) reconfiguring, progress: 0.00 (0 / 0)
 Update state (0x61) downloading, progress: 84.60 (2266216033 / 2678785722)
 Update state (0x61) downloading, progress: 85.03 (2277750369 / 2678785722)
 Update state (0x61) downloading, progress: 85.81 (2298721889 / 2678785722)
 Update state (0x61) downloading, progress: 86.48 (2316547681 / 2678785722)
 Update state (0x61) downloading, progress: 87.30 (2338567777 / 2678785722)
 Update state (0x61) downloading, progress: 88.24 (2363733601 / 2678785722)
 Update state (0x61) downloading, progress: 89.18 (2388899425 / 2678785722)
 Update state (0x61) downloading, progress: 90.24 (2417210977 / 2678785722)
 Update state (0x61) downloading, progress: 91.29 (2445522529 / 2678785722)
 Update state (0x61) downloading, progress: 92.62 (2481174113 / 2678785722)
 Update state (0x61) downloading, progress: 93.82 (2513113529 / 2678785722)
 Update state (0x61) downloading, progress: 95.07 (2546667961 / 2678785722)
 Update state (0x61) downloading, progress: 96.24 (2578125241 / 2678785722)
 Update state (0x61) downloading, progress: 97.34 (2607485369 / 2678785722)
 Update state (0x61) downloading, progress: 98.75 (2645234105 / 2678785722)
 Update state (0x61) downloading, progress: 99.92 (2676691385 / 2678785722)
 Update state (0x81) verifying update, progress: 5.72 (153259418 / 2678785722)
 Update state (0x81) verifying update, progress: 21.31 (570819368 / 2678785722)
 Update state (0x81) verifying update, progress: 36.75 (984384807 / 2678785722)
 Update state (0x81) verifying update, progress: 52.40 (1403815207 / 2678785722)
 Update state (0x81) verifying update, progress: 67.83 (1816954151 / 2678785722)
 Update state (0x81) verifying update, progress: 83.15 (2227526835 / 2678785722)
 Update state (0x81) verifying update, progress: 98.69 (2643634850 / 2678785722)
Success! App '2394010' fully installed.
./PalServer.sh -port=8211 -players=16
*****STARTING SERVER*****Password: su: Authentication failure

ecterun commented 4 months ago

EDIT:

Update the container image to latest as issue has been resolved based on fix: https://github.com/thijsvanloef/palworld-server-docker/commit/090eb098c82b2dabad59f3e9b972f1ca7a74f40f

Hello,

I have found the underlying issue, and a current work-around for anyone else running into the "(Missing file permissions)" error message.

Error:

Connecting anonymously to Steam Public...OK
Waiting for client config...OK
Waiting for user info...OK
ERROR! Failed to install app '2394010' (Missing file permissions)

Issue:

Steamcmd is using the $HOME as the base path for both redirecting stderr and the logging directory. In my case that was sending it to /root which the default container user did not have access to. This is what is being described in the missing file permissions, and it stops the updates from being installed.

This caused the set values while trying to download a game update to this:

Redirecting stderr to '/root/Steam/logs/stderr.txt'
Logging directory: '/root/Steam/logs'

which due to permissions on those directories would lead to the following error message,

ERROR! Failed to install app '2394010' (Missing file permissions)

Work Around:

The work around is to update the $HOME environment variable to a location owned by the default user (steam), in my example that is /home/steam.

I added this line to my container build command, and rebuilt the container.

-e HOME=/home/steam \

Once the container spins up, you will see it is correctly redirecting the stderr and setting the logging directory to a location owned by the default user(steam).

Redirecting stderr to '/home/steam/Steam/logs/stderr.txt'
Logging directory: '/home/steam/Steam/logs'

And the update is installed successfully.

Connecting anonymously to Steam Public...OK
Waiting for client config...OK
Waiting for user info...OK
 Update state (0x3) reconfiguring, progress: 0.00 (0 / 0)
 Update state (0x3) reconfiguring, progress: 0.00 (0 / 0)
 Update state (0x3) reconfiguring, progress: 0.00 (0 / 0)
 Update state (0x5) verifying install, progress: 5.94 (172097507 / 2895752351)
 Update state (0x5) verifying install, progress: 17.49 (506425740 / 2895752351)
 Update state (0x5) verifying install, progress: 26.29 (761229708 / 2895752351)
 Update state (0x5) verifying install, progress: 37.82 (1095103371 / 2895752351)
 Update state (0x5) verifying install, progress: 48.57 (1406530443 / 2895752351)
 Update state (0x5) verifying install, progress: 58.86 (1704326027 / 2895752351)
 Update state (0x5) verifying install, progress: 69.50 (2012607371 / 2895752351)
 Update state (0x5) verifying install, progress: 80.68 (2336148247 / 2895752351)
 Update state (0x5) verifying install, progress: 92.36 (2674661638 / 2895752351)
 Update state (0x61) downloading, progress: 3.36 (90059830 / 2678768432)
 Update state (0x41) staging, progress: 41.48 (1111128548 / 2678768432)
 Update state (0x41) staging, progress: 90.87 (2434275510 / 2678768432)
 Update state (0x41) staging, progress: 100.00 (2678768432 / 2678768432)
 Update state (0x81) verifying update, progress: 15.28 (409328878 / 2678768432)
 Update state (0x81) verifying update, progress: 31.09 (832737653 / 2678768432)
 Update state (0x81) verifying update, progress: 45.32 (1214012564 / 2678768432)
 Update state (0x81) verifying update, progress: 61.06 (1635540116 / 2678768432)
 Update state (0x81) verifying update, progress: 76.65 (2053335348 / 2678768432)
 Update state (0x81) verifying update, progress: 92.50 (2477949802 / 2678768432)
 Update state (0x101) committing, progress: 100.00 (2678768432 / 2678768432)
 Update state (0x101) committing, progress: 100.00 (2678768432 / 2678768432)
 Update state (0x101) committing, progress: 100.00 (2678768432 / 2678768432)
 Update state (0x101) committing, progress: 100.00 (2678768432 / 2678768432)
 Update state (0x101) committing, progress: 100.00 (2678768432 / 2678768432)
Success! App '2394010' fully installed.

Context

Source going over same issue, with a different application, this is what lead me to my work around: https://github.com/ValveSoftware/steam-for-linux/issues/10979#issuecomment-2174427130

thijsvanloef commented 4 months ago

That is really wierd as HOME=/home/steam is included by default in the Dockerfile, and has been for 5 months https://github.com/thijsvanloef/palworld-server-docker/commit/090eb098c82b2dabad59f3e9b972f1ca7a74f40f

ecterun commented 4 months ago

That would be because the image I, and I imagine @dichromatic were using was very outdated. I think you are probably good to close this now, as if someone uses the latest image it will be resolved, and if someone runs into this issue in the future they have my previous comment describing whats going on, but in the end they should update to the latest image of the container.

thijsvanloef / palworld-server-docker