Server never starts when using Docker + Volumes

[svanvliet]

Trying to get a persistent volume attached so that I can use this image for hosting. When attaching to a local volume (and also a volume in Azure), it hangs on the "Downloading Bedrock server version 1.17.10.04 ..." step and never progresses.

Output from console (Azure Container Instance):

time="2021-08-04T19:05:30Z" level=debug msg="Using /data to match uid and gid"
time="2021-08-04T19:05:30Z" level=debug msg="Resolved UID=0 from match path"
time="2021-08-04T19:05:30Z" level=debug msg="Resolved GID=0 from match path"
Looking up latest version...
time="2021-08-04T19:05:30Z" level=debug msg="stdin is detached, so forwarding is disabled"
Downloading Bedrock server version 1.17.10.04 ...

Here's my docker-compose.yaml file:

version: "3.4"

services:
    bedrock-server:
        image: itzg/minecraft-bedrock-server
        volumes:
            - mcdata:/data
        environment: 
            EULA: "TRUE"
            GAMEMODE: survival
            DIFFICULTY: normal
        domainname: "bedrock"
        ports: 
            - "80:80"
            - "19132:19132/udp"
        stdin_open: true
        tty: true
        sysctls:
            net.ipv4.ip_local_port_range: 39132 39133
        deploy:
            resources:
                    reservations:
                        cpus: "2"
                        memory: 4G

volumes:
    mcdata:
        driver: azure_file
        driver_opts:
            share_name: mcdata
            storage_account_name: azurestorageaccount 

When I remove the custom volume from this compose file, it all works as expected (in Azure and on local Docker Desktop). Thanks for the help!

[itzg]

I don't have any ideas offhand. Can you add DEBUG: "true" to your environment and see if that reveals any more clues? It sounds like it's a filesystem ownership/write-permission mismatch, but I don't have experience with Azure deployments.

[svanvliet]

I think I may have found something. I tried deleting files from the persistent share in Azure while the container was running, and got an error message saying the directory was in use. I watched in the Azure Storage Explorer and notice that the behavior_packs folder continued having new versions added. I checked in Cloud Shell and confirmed, but also notice that the file operations were happening EXTREMELY slowly.

I think it's stalling on the unzip step at line 121. It seems that the unzip process between the container's ephemeral instance storage (which downloads the file quite quickly) and the mounted Azure Storage volume is doggedly slow. To test this, I mounted the file via SMB and tried manually copying the files from my machine after unzipping locally. Finder (macOS) said it will take ~7 hours to copy!

I wonder if (a) downloading the zip file directly to the /data volume would actually mitigate this; (b) if this is just a problem with the way I've provisioned the storage in Azure; or (c) if there's a known issue with Docker containers performing unzip/copy operations over managed volumes. I'll test (b) now by creating a new share with the "Premium" SKU in Azure and see if that helps. Stay tuned.

[itzg]

That's an interesting idea about downloading the zip directory to the /data volume. I know some file operations like mv/cp are optimized when staying on the same filesystem. I could see unzipping going either way since it wouldn't involve a direct block transfer since the content has to expand via CPU/memory.

Yes, interested to find out if the volume class makes a difference. I also know with GCP volumes they would throttle I/O based upon how large the volume was. That might be another (possibly cheaper?) variable to explore, if Azure does the same.

[djryan012]

I also seem to be having a similar issue even with restarting my docker container. It hangs up on " Looking for latest version" I use docker composes file.

version: '3.4'

services:
  Minecraft:
    image: itzg/minecraft-bedrock-server
    environment:
      EULA: "TRUE"
      GAMEMODE: survival
      DIFFICULTY: normal
      DEBUG: "true"
    ports:
      - 19132:19132/udp
    volumes:
      - /volume2/mc_bedrock:/data
    stdin_open: true
    tty: true

volumes:
    MineCraft:

Here is my debug log from latest try.

2021-09-04T16:45:27.436303262Z DEBU[0000] Using /data to match uid and gid             
,2021-09-04T16:45:27.436375626Z DEBU[0000] Resolved UID=999 from match path             
,2021-09-04T16:45:27.436393850Z DEBU[0000] Resolved GID=999 from match path             
,2021-09-04T16:45:27.468868273Z + curlArgs=(-v)
,2021-09-04T16:45:27.469202444Z ++ id -a
,2021-09-04T16:45:27.474961245Z ++ ls -ld /data
,2021-09-04T16:45:27.477814854Z + echo 'DEBUG: running as uid=999 gid=999 groups=999 with drwxrwxrwx 16 999 999 4096 Sep  4 16:38 /data'
,2021-09-04T16:45:27.477845470Z DEBUG: running as uid=999 gid=999 groups=999 with drwxrwxrwx 16 999 999 4096 Sep  4 16:38 /data
,2021-09-04T16:45:27.477855111Z + downloadPage=https://www.minecraft.net/en-us/download/server/bedrock
,2021-09-04T16:45:27.477925703Z + [[ TRUE != TRUE ]]
,2021-09-04T16:45:27.477939619Z + case ${VERSION^^} in
,2021-09-04T16:45:27.477999324Z + echo 'Looking up latest version...'
,2021-09-04T16:45:27.478015137Z Looking up latest version...
,2021-09-04T16:45:27.478094865Z + for a in data-platform
,2021-09-04T16:45:27.478107838Z + for i in {1..3}
,2021-09-04T16:45:27.478687110Z ++ jq -r '.[0].href'
,2021-09-04T16:45:27.478766477Z ++ restify --user-agent=itzg/minecraft-bedrock-server --headers 'accept-language:*' --attribute=data-platform=serverBedrockLinux https://www.minecraft.net/en-us/download/server/bedrock
,2021-09-04T16:45:37.512820434Z ++ echo ''
,2021-09-04T16:45:37.513072588Z + DOWNLOAD_URL=
,2021-09-04T16:45:37.513091549Z + [[ -n '' ]]
,2021-09-04T16:45:37.513103955Z + sleep 1
,2021-09-04T16:45:38.517053421Z + for i in {1..3}
,2021-09-04T16:45:38.517671483Z ++ restify --user-agent=itzg/minecraft-bedrock-server --headers 'accept-language:*' --attribute=data-platform=serverBedrockLinux https://www.minecraft.net/en-us/download/server/bedrock
,2021-09-04T16:45:38.517689211Z ++ jq -r '.[0].href'
,2021-09-04T16:45:48.541874872Z ++ echo ''
,2021-09-04T16:45:48.542090761Z + DOWNLOAD_URL=
,2021-09-04T16:45:48.542108087Z + [[ -n '' ]]
,2021-09-04T16:45:48.542117183Z + sleep 1
,2021-09-04T16:45:49.543773802Z + for i in {1..3}
,2021-09-04T16:45:49.544425953Z ++ restify --user-agent=itzg/minecraft-bedrock-server --headers 'accept-language:*' --attribute=data-platform=serverBedrockLinux https://www.minecraft.net/en-us/download/server/bedrock
,2021-09-04T16:45:49.544445367Z ++ jq -r '.[0].href'
,2021-09-04T16:45:59.562772112Z ++ echo ''
,2021-09-04T16:45:59.562962399Z + DOWNLOAD_URL=
,2021-09-04T16:45:59.562984137Z + [[ -n '' ]]
,2021-09-04T16:45:59.562996348Z + sleep 1
,2021-09-04T16:46:00.564704802Z + [[ -z '' ]]
,2021-09-04T16:46:00.564997016Z ++ curl -s https://mc-bds-helper.vercel.app/api/latest
,2021-09-04T16:46:10.629226679Z + DOWNLOAD_URL=

unfortunately this is the extent of which I can help.

[djryan012]

I just added latest to the end of image pull "image: itzg/minecraft-bedrock-server:latest" in the compose file and docker container started with no issues. Here is full debug log with with latest added. Hope this helps explain my sistuation.

DEBU[0000] Using /data to match uid and gid             
,DEBU[0000] Resolved UID=999 from match path             
,DEBU[0000] Resolved GID=999 from match path             
,+ curlArgs=(-v)
,++ id -a
,++ ls -ld /data
,+ echo 'DEBUG: running as uid=999 gid=999 groups=999 with drwxrwxrwx 16 999 999 4096 Sep  4 16:38 /data'
,DEBUG: running as uid=999 gid=999 groups=999 with drwxrwxrwx 16 999 999 4096 Sep  4 16:38 /data
,+ downloadPage=https://www.minecraft.net/en-us/download/server/bedrock
,+ [[ TRUE != TRUE ]]
,+ case ${VERSION^^} in
,+ echo 'Looking up latest version...'
,Looking up latest version...
,+ for a in data-platform
,+ for i in {1..3}
,++ restify --user-agent=itzg/minecraft-bedrock-server --headers 'accept-language:*' --attribute=data-platform=serverBedrockLinux https://www.minecraft.net/en-us/download/server/bedrock
,++ jq -r '.[0].href'
,+ DOWNLOAD_URL=https://minecraft.azureedge.net/bin-linux/bedrock-server-1.17.11.01.zip
,+ [[ -n https://minecraft.azureedge.net/bin-linux/bedrock-server-1.17.11.01.zip ]]
,+ break 2
,+ [[ -z https://minecraft.azureedge.net/bin-linux/bedrock-server-1.17.11.01.zip ]]
,+ [[ https://minecraft.azureedge.net/bin-linux/bedrock-server-1.17.11.01.zip =~ http.*/.*-(.*)\.zip ]]
,+ VERSION=1.17.11.01
,+ rm -f restify.err
,+ '[' '!' -f bedrock_server-1.17.11.01 ']'
,+ '[' -n '' ']'
,+ '[' -n '' ']'
,+ '[' -n '' ']'
,+ allowListUsers=
,+ '[' -n '' ']'
,+ set-property --file server.properties --bulk /etc/bds-property-definitions.json
,+ export LD_LIBRARY_PATH=.
,+ LD_LIBRARY_PATH=.
,+ echo 'Starting Bedrock server...'
,Starting Bedrock server...
,+ exec ./bedrock_server-1.17.11.01
,NO LOG FILE! - Failed to up file descriptor limit Operation not permitted
,NO LOG FILE! - setting up server logging...
,[2021-09-04 17:45:43 INFO] Starting Server
,[2021-09-04 17:45:43 INFO] Version 1.17.11.01
,[2021-09-04 17:45:43 INFO] Session ID 97d40f72-7832-4125-xxxxxxxxxxxxxxxxxxxx
,[2021-09-04 17:45:43 INFO] Level Name: Mark and Davids
,[2021-09-04 17:45:43 INFO] Game mode: 0 Survival
,[2021-09-04 17:45:43 INFO] Difficulty: 2 NORMAL
,[2021-09-04 17:45:43 INFO] Content logging enabled. Writing log to: ContentLog__Saturday__2021_September_04__17_45_43
,[INFO] opening worlds/Mark and Davids/db
,[INFO] IPv4 supported, port: 19132
,[INFO] IPv6 not supported
,[INFO] IPv4 supported, port: 37112
,[INFO] IPv6 not supported
,[INFO] Server started.

[itzg]

@djryan012 yours seems to be a temporary networking issue since this fallback endpoint is running fine but yours failed

https://mc-bds-helper.vercel.app/api/latest

[geekpete]

If everyone's docker containers based on this one project are all hammering the url, any chance there's rate limiting or abuse protection that just blocks these curls? Any way to customize/override the url it's checking?

[geekpete]

Right, so pinning the version avoids the lookup call to check for latest, seems to be more reliable?

in your ENVIRONMENT section, use:

VERSION: "1.17.11.01"

And as above, use https://mc-bds-helper.vercel.app/api/latest to check the latest version to pin to so you can manually upgrade. At least this should mean as long as the download is available, and the file is fetchable (unsure how many versions they keep online,etc), then your Minecraft environment can be a bit more reliable and stay up.

[itzg]

If everyone's docker containers based on this one project are all hammering the url, any chance there's rate limiting or abuse protection that just blocks these curls?

That's a good theory except that a) minecraft.net is very unlikely to be rate limiting except for DDoS attack and b) the original issue also happened with my endpoint I created and deployed at https://mc-bds-helper.vercel.app/api/latest. So, two totally different web services rate limiting is highly unlikely. Instead, that's why I suspected networking issues.

Any way to customize/override the url it's checking?

I open to suggestions of a better URL to check. I hate having to scrape minecraft.net and it was a shame I had to implement by own web service.

[geekpete]

Right, that blows that theory out of the water mostly I think. So likely need to debug that restify command why it's failing specifically.