search

itzg / docker-minecraft-server

Docker image that provides a Minecraft Server that will automatically download selected version at startup
https://docker-minecraft-server.readthedocs.io/
Apache License 2.0
9.2k stars 1.52k forks source link

CurseForge modpack not pulled from URL #392

Closed pixtim closed 4 years ago

pixtim commented 4 years ago

Hi Itzg. Thanks for the awesome docker image!

I'm trying to spin up a container from a CurseForge URL. I've followed the steps in the README but it looks like the TYPE and CF_SERVER_MOD environment variables are being ignored. A vanilla server is deployed instead of a modded one. I'm hoping that I've missed something small.

Steps to reproduce:

  1. Pull the image

    docker pull itzg/minecraft-server

  2. Create a CurseForge server container

    docker run -d -it -e EULA=TRUE -p 25565:25565 --name mc itzg/minecraft-server \
-e VERSION=1.12.2 \
-e TYPE=CURSEFORGE \
-e CF_SERVER_MOD=https://www.curseforge.com/minecraft/modpacks/rlcraft

  3. Start the container

    docker container start mc

    See logs.txt

itzg commented 4 years ago

Ah, the -e arguments need to come before the image name, so try:

docker run -d -it -e EULA=TRUE -p 25565:25565 --name mc \
-e VERSION=1.12.2 \
-e TYPE=CURSEFORGE \
-e CF_SERVER_MOD=https://www.curseforge.com/minecraft/modpacks/rlcraft \
itzg/minecraft-server
alessandro-massarenti commented 4 years ago

Ah, the -e arguments need to come before the image name, so try:

docker run -d -it -e EULA=TRUE -p 25565:25565 --name mc \
-e VERSION=1.12.2 \
-e TYPE=CURSEFORGE \
-e CF_SERVER_MOD=https://www.curseforge.com/minecraft/modpacks/rlcraft \
itzg/minecraft-server

Hi, I've been trying for days to start a Curseforge server in vain.

I tried first by following the directions and composing a docker string for a custom RLCraft server, but it never started because it said that probably there is no server executable in the link I gave it.

So I decided to try any example of docker-compose contained in the examples folder of your repo; also there is nothing to do but with a different error:

Downloading FTB modpack...
  https://www.curseforge.com/minecraft/modpacks/rlcraft -> /data/modpacks.zip
Unpacking FTB server modpack /data/modpacks.zip ...
.unzip: short read

I then saw today here among the issues the one I'm replying to now, and I tried this docker line that you just posted, but also from the same error.( which i posted below)

myuser@mydomain:~$ sudo docker run --rm -it -e EULA=TRUE -p 25565:25565 --name mc -e VERS
raft-server
usermod: no changes
Running as uid=1000 gid=1000 with /data as 'drwxrwxrwx    2 1000     1000          4096 Sep 27 15:11
Checking version information.
Checking type information.
Looking for Feed-The-Beast / CurseForge server modpack.
Downloading FTB modpack...
  https://www.curseforge.com/minecraft/modpacks/rlcraft -> /data/modpacks.zip
Unpacking FTB server modpack /data/modpacks.zip ...
.unzip: short read

Please make sure you are using the server version of the FTB modpack!

PS: yes i changed the-detach with a --remove to try it without cluttering my server

thank you in advance, let me know if i should open another issue :)

itzg commented 4 years ago

I'll have to debug what the modpack zip contains, because even when I navigate to

https://www.curseforge.com/minecraft/modpacks/rlcraft/files

and grab the server download URL from the "Additional Files", https://www.curseforge.com/minecraft/modpacks/rlcraft/download/2791783 then it still reports the unzip "short read" error.

There might be an HTTP redirect that needs to be accounted for...

itzg commented 4 years ago

Oh, CurseForge may have updated things to defeat non-human downloads since it now seems to through an adfly type page:

image

@pixtim if you don't mind us piggybacking on this issue, I could adjust the title for "RLCraft" just in case it's something specific to that.

alessandro-massarenti commented 4 years ago

Oh, CurseForge may have updated things to defeat non-human downloads since it now seems to through an adfly type page:

image

@pixtim if you don't mind us piggybacking on this issue, I could adjust the title for "RLCraft" just in case it's something specific to that.

Another thing, if it can be useful.

Even giving him the zip downloaded from the website at the end he says "are you sure it's a file server?" (I don't have this log about it now though)

Maybe it's me that I did something wrong but not knowing I imagine that it could be useful as information.

pixtim commented 4 years ago

@itzg Thanks for the help with the -e argument. 'Was rather silly of me. I got the mod-pack version of the image running and started running into the same issues as @alessandro-massarenti . We can close this issue if you like - the image is working.

I managed to get the RLCraft mod-pack server running by downloading the server pack manually and copying it into a mounted data volume for the container. I've pasted my documentation below so that it can be replicated.

README

Guide

See https://hub.docker.com/r/itzg/minecraft-server/ See https://github.com/AllTheMods/Server-Scripts

Installing the Docker Image

Creating a Vanilla Server

Creating a CurseForge Server

  1. Create a docker volume directory on the host OS.
  2. Download the server mod-pack from CurseForge and put it inside the volume directory.
  3. Download the server scripts from Github and put it inside the FeedTheBeast directory inside volume directory.
  4. Create a docker container and run it: 
    
docker run -d -it -e EULA=TRUE -p 25565:25565 \
-v /home/tim/docker/volume:/data \
-e TYPE=CURSEFORGE \
-e CF_SERVER_MOD=RLCraft.zip \
-e MANIFEST=/data/FeedTheBeast/manifest.json \
--name mc itzg/minecraft-server

docker container start mc



## Modded Client Installation

- Download the client mod-pack archive from https://www.curseforge.com/minecraft/mc-mods
- Create a new instance in MultiMC using the Twitch project URL for the modpack.
- Add OptiFine manually and install a shader pack.
pixtim commented 4 years ago

Also, the forge server prints that the EULA is not accepted even though it's set to eula=TRUE in the eula.txt file. Looks like it's case sensitive. Setting it to eula=true fixes the issue.

pixtim commented 4 years ago

Okay looks like it's spinning up a vanilla server. I'm investigating. :thinking:

alessandro-massarenti commented 4 years ago

still not working to me.

It says "are you sure that this file is a server file?"

even by getting it your(@pixtim) docker string and the right server zip

itzg commented 4 years ago

There's something wrong with the RL Craft server packs. I checked 2.6.2 and 2.5 and they're both missing the server startup script, ServerStart.sh and installer jar at the top level:

image

I would recommend reporting an issue against the modpack; however, since I personally don't use modpacks much at all, I'm not sure how that's done.

Meanwhile, I'll explore adding an option to generically combine the unzipped content of a modpack like this with a TYPE=FORGE VERSION=1.12.2 invocation.

pixtim commented 4 years ago

Looks like the zip file i'm passing in manually has the same .unzip: short read issue. I'm going to try and upload the file to my server manually and chown, chmod it.

Thundernerd commented 4 years ago

This may or may not help you regarding the downloading of the server file but it turns out that the following download link https://www.curseforge.com/minecraft/modpacks/skyfactory-4/files/2725984 turns into https://media.forgecdn.net/files/2725/984/SkyFactory_4_Server_4.0.8.zip which it is able to download without any issues.

I also noticed that the numbers on the end of the first URL are split into two sections. The first being the first four digits and the second being the last three digits. The only thing missing then is the filename which is also required.

@itzg it is definitely not just RLCraft related! I've tried it with SevTech and SkyFactory 4 as well and had the same issue.

itzg commented 4 years ago

@Thundernerd thank for that analysis! I was also trying to see what the Twitch client was resolving when using the server download, but the URL went away too fast :)

itzg commented 4 years ago

As a solution for RL Craft (and other server-less server packs) I have add a GENERIC_PACK option that can be used to stitch together a CurseForge server, such as:

version: '3.7'

services:
  rlcraft:
    image: itzg/minecraft-server
    environment: 
      EULA: "TRUE"
      TYPE: FORGE
      VERSION: 1.12.2
      FORGEVERSION: 14.23.5.2838
      # Can be a URL or a container-local path to a zip containing mods, plugins, and other directories
      # The checksum of the file is evaluated to ensure it is unpacked only once into the /data directory.
      GENERIC_PACK: /modpacks/RLCraft+Server+Pack+1.12.2+-+Beta+v2.6.3.zip
      # Number of directories to strip from the unzipped content, default is 1
      GENERIC_PACK_STRIP_DIRS: "1"
    ports: 
      - 25565:25565
    volumes:
      - ./data:/data
      - ./modpacks:/modpacks

If a couple people could test this out, that would be great. I will then add it to the README as an official option.

itzg commented 4 years ago

@Thundernerd , how were you able to determine the URL conversion it is doing? Specifically I'm wondering if the algorithm can be generalized for less and more file ID digits where it is always the first four digits in first path part and then remaining digits in the other part?

itzg commented 4 years ago

Actually might have answered the first question by doing a network trace in Chrome. It seems to start with an "edge CDN" URL like https://edge.forgecdn.net/files/2725/984/SkyFactory_4_Server_4.0.8.zip and then that 302's to the media.forgecdn.net one.

Thundernerd commented 4 years ago

That's exactly what I did :). I think it's always the last three go into the second segment but I'm not exactly sure about that. I thought I saw a mod that had five digits in the first segment but I couldn't find it anymore.

pixtim commented 4 years ago

@itzg I can confirm that the new GENERIC_PACK option works. I've added docker-compose to my VPS and now I can spin up the RL-craft server by just creating the volume directories, copying over the modpack and running compose. Thanks! This so awesome! :beers:

denzuko commented 4 years ago

GENERIC_PACK seems to be helping. Ran into this issue trying to get ChoosenArchitect's Bounds 3.0 modpack setup. Currently though the container dies after a time out and shows the following:


usermod: no changes

Running as uid=1000 gid=1000 with /data as 'drwxrwxrwx    5 1000     1000          4096 Oct 11 02:37 /data'

Checking version information.

Checking type information.

Checking Forge version information.

Downloading generic pack from https://media.forgecdn.net/files/2623/86/Bounds+3.0+Server+Files.zip ...

The docker-compose.yml I'm using:

---
version: "3.7"

volumes:
  mc-data:

networks:
  default:
    external: true
    name: 'public'

services:
  minecraft-server:
    ports:
      - "25565:25565"

    environment:
      EULA: 'true'
      VERSION: 1.12.2
      DIFFICULTY: normal
      WHITELIST: zekodun,denzuko
      OPS: zekodun
      TYPE: FORGE
      GENERIC_PACK: https://media.forgecdn.net/files/2623/86/Bounds+3.0+Server+Files.zip
      ALLOW_NETHER: 'true'
      ANNOUNCE_PLAYER_ACHIEVEMENTS: "true"
      ENABLE_COMMAND_BLOCK: "true"
      FORCE_GAMEMODE: "true"
      GENERATE_STRUCTURES: "true"
      SPAWN_ANIMALS: "true"
      SPAWN_MONSTERS: "true"
      SPAWN_NPCS: "true"
      MODE: "survival"
      PVP: "false"
      SEED: 4031384495743822299
      ENABLE_QUERY: "true"
      VIEW_DISTANCE: "10"
      MEMORY: "3G"
      MAX_MEMORY: "3G"
      INIT_MEMORY: "512m"
      GUI: "FALSE"
      CONSOLE: "FALSE"

    networks:
      - default

    image: itzg/minecraft-server:latest
    tty: true
    stdin_open: true
    shm_size: 128M
    deploy:
      resources:
        limits:
          memory: 3.5G
      restart_policy:
        condition: on-failure
    volumes:
      - mc-data:/data

...

Portainer shows the Last output as:

 Traceback (most recent call last): File "/usr/bin/mcstatus", line 11, in <module> sys.exit(cli()) File "/usr/lib/python2.7/site-packages/click/core.py", line 764, in __call__ return self.main(*args, **kwargs) File "/usr/lib/python2.7/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/usr/lib/python2.7/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/lib/python2.7/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/lib/python2.7/site-packages/click/core.py", line 555, in invoke return callback(*args, **kwargs) File "/usr/lib/python2.7/site-packages/mcstatus/scripts/mcstatus.py", line 49, in ping click.echo("{}ms".format(server.ping())) File "/usr/lib/python2.7/site-packages/mcstatus/server.py", line 36, in ping connection = TCPSocketConnection((self.host, self.port)) File "/usr/lib/python2.7/site-packages/mcstatus/protocol/connection.py", line 129, in __init__ self.socket = socket.create_connection(addr, timeout=timeout) File "/usr/lib/python2.7/socket.py", line 575, in create_connection raise err socket.error: [Errno 99] Address not available

Don't want to put the blame directly on mcstatus but is there anyway to control the timeout on that? Bounds is a 158Mb pack and on my network would take about 8 minutes or more to download.

denzuko commented 4 years ago

found my own answer... silly me should have thought about this one too. Overwritten the heathcheck by setting the interval and start_period to later in the sequence.

....
healthcheck:
      test: ['CMD', 'mcstatus', 'localhost:25565','ping']
      interval: 10m
      timeout: 10s
      retries: 3
      start_period: 10m
....

Still waiting on it to complete the download but at lease now the health check is not killing the container too soon. Might one suggest changing the Dockerfile to include --start-period=5m? to offset the startup time for any mod pack or other downloads to complete?

denzuko commented 4 years ago

Added, Pull Request #397 which includes the suggested change.

Also found out that using the url method when the container restarts it always downloads the modpack. Some use cases might need that but perhaps only when the mods folder does not exist, or the shasum is out of sync, or when REMOVE_OLD_MODS is set?

denzuko commented 4 years ago

Pull Request #398 implements the above suggestion.

itzg commented 4 years ago

Sorry @denzuko , I read your PR and these comments out of order, so we both arrived at the same idea for overriding the health check startup interval.

obsessionwithfire commented 4 years ago

I'm running into a similar/related issue

I'm creating my container using the following (info has been sanitized)

docker run -itd --name minecraft\
  -v /opt/minecraft:/data\
  -e MANIFEST=/data/FeedTheBeast/manifest.json\
  -e VERSION=1.12.2\
  -e TYPE=CURSEFORGE\
  -e CF_SERVER_MOD=https://media.forgecdn.net/files/2756/981/ATM3-5.12.3_Server-FULL.zip\
  -p 25565:25565\
  -e EULA=TRUE\
  -e VIRTUAL_HOST=mc.domain.com \
  -e LETSENCRYPT_HOST=mc.domain.com \
  -e LETSENCRYPT_EMAIL=admin@domain.com \
  --restart=always\
    itzg/minecraft-server

Which results in the following

➜  ~ docker logs minecraft -f
usermod: no changes
Running as uid=1000 gid=1000 with /data as 'drwxrwxr-x    2 1000     1000          4096 Oct 23 07:31 /data'
Checking version information.
Checking type information.
Looking for Feed-The-Beast / CurseForge server modpack.
Downloading FTB modpack...
  https://media.forgecdn.net/files/2756/981/ATM3-5.12.3_Server-FULL.zip -> /data/981.zip
Unpacking FTB server modpack /data/981.zip ...
...............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
usermod: no changes
Running as uid=1000 gid=1000 with /data as 'drwxrwxr-x    3 1000     1000          4096 Oct 23 07:32 /data'
Checking version information.
Checking type information.
Looking for Feed-The-Beast / CurseForge server modpack.
usermod: no changes
Running as uid=1000 gid=1000 with /data as 'drwxrwxr-x    3 1000     1000          4096 Oct 23 07:32 /data'
Checking version information.
Checking type information.
Looking for Feed-The-Beast / CurseForge server modpack.
usermod: no changes
Running as uid=1000 gid=1000 with /data as 'drwxrwxr-x    3 1000     1000          4096 Oct 23 07:32 /data'
Checking version information.
Checking type information.
Looking for Feed-The-Beast / CurseForge server modpack.
usermod: no changes
Running as uid=1000 gid=1000 with /data as 'drwxrwxr-x    3 1000     1000          4096 Oct 23 07:32 /data'
Checking version information.
Checking type information.
Looking for Feed-The-Beast / CurseForge server modpack.

Any help would be greatly appreciated.

itzg commented 4 years ago

@obsessionwithfire do you mind opening a new issue? Yours is definitely in the general realm of FTB/CurseForge challenges, but seems to be different since the startup logic is quietly failing at the point:

Looking for Feed-The-Beast / CurseForge server modpack.

I probably have some missing error checking around there.

obsessionwithfire commented 4 years ago

@itzg. Done, thanks for your help.

ChipWolf commented 4 years ago

If REMOVE_OLD_MODS is true and the GENERIC_PACK's sum doesn't equal the last sum (if one exists), it should probably remove any file which the generic pack introduced on the last run. I assume this could be implemented by creating a manifest of file paths after the pack is downloaded. This would allow for any generic files to be replaced between runs with no fear of files deleted from the generic pack left dangling.

ChipWolf commented 4 years ago

@denzuko I modified this behavior in itzg/docker-minecraft-server#491

Lucria commented 4 years ago

@pixtim Can I ask how did you manage to get your server to work? I have tried with this current docker-compose.yml file but the server spun up was only a vanilla server. Is it okay if you can share your docker-compose file as well?

This is what my docker-compose file looks like:

version: '3.8'

services:
    mc:
        image: itzg/minecraft-server
        ports:
            - 25565:25565
        volumes:
            - ./modpacks:/modpacks
        environment:
            TYPE: "FORGE"
            VERSION: "1.12.2"
            FORGEVERSION: "14.23.5.2838"

            GENERIC_PACK: "/modpacks/RLCraft+Server+Pack+1.12.2+-+Beta+v2.8.2.zip"
            GENERIC_PACK_STRIP_DIRS: "1"

            INIT_MEMORY: "1G"
            MAX_MEMORY: "12G"
            EULA: "TRUE"
            MAX_TICK_TIME: -1
            REPLACE_ENV_VARIABLES: "TRUE"
            ONLINE_MODE: "FALSE"
            ALLOW_FLIGHT: "TRUE"
            DIFFICULTY: "3"
            VIEW_DISTANCE: "6"
pixtim commented 4 years ago

Hi @Lucria

My compose file:

version: '3.7'

services:
  mc-pix:
    image: itzg/minecraft-server
    deploy:
      restart_policy:
        condition: any
        max_attempts: 3
        window: 120s
    environment:
      EULA: "TRUE"
      TYPE: FORGE
      VERSION: 1.12.2
      FORGEVERSION: 14.23.5.2838
      MEMORY: 5G
      GENERIC_PACK: <URL TO YOUR ZIP FILE>
      GENERIC_PACK_STRIP_DIRS: "1" # This strips off the directory of the zip-file itself.
      DIFFICULTY: easy
      OPS: pixtim
      MODE: survival
      PVP: "true"
      SEED: 1372204
      VIEW_DISTANCE: 4
      MAX_WORLD_SIZE: 10000
      ENABLE_RCON: "true"
      RCON_PASSWORD: secret
      RCON_PORT: 25575
    ports:
      - 25565:25565
      - 25575:25575
      - 8123:8123
    volumes:
      - ./data:/data
      - ./modpacks:/modpacks
  mc-rcon-admin:
    image: itzg/rcon
    deploy:
      restart_policy:
        condition: any
        max_attempts: 3
        window: 120s
    ports:
      - 4326:4326
      - 4327:4327
    environment:
      RWA_USERNAME: admin
      RWA_PASSWORD: secret
      RWA_RCON_HOST: mc-pix
      RWA_RCON_PASSWORD: secret

Replace <URL TO YOUR ZIP FILE> with the URL to a ZIP file that contains your mods. The zip must contain a mods directory that has the Forge mods in it. For example:

- Pix 5.0 server.zip <-- The zip file itself
  - mods
    - mod-1.jar
    - mod-2.jar
    ...
ChipWolf commented 4 years ago

@pixtim GENERIC_PACK_STRIP_DIRS no longer exists, I got rid of it

maxaudron commented 4 years ago

@ChipWolf for what reason did you remove the strip variable, i just ran into the same problem mentioned here and it would solve it. What's the alternative?

ChipWolf commented 4 years ago

@maxaudron GENERIC_PACK_URL grabs a zip from a URL on startup and recursively adds it to the root of the server If that zip has changed in any way since the last startup, it'll remove the files added by the old generic pack, then recursively adds the new generic pack.

So, for example: if your generic pack has a mods folder at its root, it'll add the mods folder to the root of the server, or if one exists; it'll merge with it

Lohann commented 3 years ago

This is how I run the RLCraft:

mkdir ~/minecraft-server
mkdir ~/modpacks

mv RLCraft+Server+Pack+1.12.2+-+Beta+v2.8.2.zip ~/modpacks/RLCraft.zip

docker run -d --rm \
  -p 25565:25565 \
  --name mc \
  -e EULA=TRUE \
  -e TYPE=FORGE \
  -e VERSION=1.12.2 \
  -e FORGEVERSION=14.23.5.2838 \
  -e GENERIC_PACK=/modpacks/RLCraft.zip \
  -e GENERIC_PACK_STRIP_DIRS=1 \
  -e INIT_MEMORY=1G \
  -e MAX_MEMORY=12G \
  -e MAX_TICK_TIME=-1 \
  -e REPLACE_ENV_VARIABLES=TRUE \
  -e ONLINE_MODE=FALSE \
  -e ALLOW_FLIGHT=TRUE \
  -e DIFFICULTY=3 \
  -e VIEW_DISTANCE=6 \
  -v ~/minecraft-server:/data \
  -v ~/modpacks:/modpacks \
  itzg/minecraft-server:1.9.0

docker logs -f mc