[Feature Request] Download server as part of container build

[mindflay3r]

Have you read the Important information text above

• [X] Yes i did

Describe the feature

Hiya Jammsen & co. I noticed when trying to build locally that part of the runtime is downloading the latest version of the server.

Interesting! Respectfully, this seems like a container anti-pattern to me. To quote some nerd on Reddit:

Docker is meant for "immutable" infrastructure. Immutable in this context means you don't change it after it's built. If you need to change something, you replace the whole thing.

src: https://www.reddit.com/r/selfhosted/comments/18c5gmh/comment/kc8yxat/

What I would propose you and co (or me if you want the help!) do is:

• Make server installation part of the container build process (Dockerfile)
• Grab the version of the downloaded server using RCON (TODO, is this necessary/possible?)
• Extract that as part of your CI and label the container with that semver. And mark it latest as ya do.

Once all this is done, cold-start times will be HUGELY reduced, builds are 'hermetic', and, maybe most importantly, you'll be creating a history of all the main server versions. This could be useful for posterity.

Additional information

• [X] Would you be willing to help implement this feature?

Final checks

• [X] I understand that this is a feature request function. I also understand that if i use this to report a bug report, that my issue might be un-answered closed.

[Callum027]

One issue with bundling the server software is that it would dramatically increase the size of the container image, as it is about 2GB at rest.

In addition, that would put the burden of updating Palworld to the latest version on the container developer. Sure, CI jobs could update the container periodically, but it would never be up to date as updating Palworld locally.

Another issue is that it would tie container and Palworld updates together, making it more difficult for users to avoid updating the container if they wish to keep the same version, but use the latest version of Palworld.

[jammsen]

Hey @Callum027 and @mindflay3r

Hiya Jammsen & co. I noticed when trying to build locally that part of the runtime is downloading the latest version of the server.

Hey, not sure what you mean by build locally, but yes the gameserver is on purpose downloaded when the user has started the image and did some kind of a volume definition.

Interesting! Respectfully, this seems like a container anti-pattern to me. To quote some nerd on Reddit:

Docker is meant for "immutable" infrastructure. Immutable in this context means you don't change it after it's built. If you need to change something, you replace the whole thing.

src: https://www.reddit.com/r/selfhosted/comments/18c5gmh/comment/kc8yxat/

"Immutable in this context means you don't change it after it's built." - This is acutally what i do, my image doesnt change, only the assets it provides do.

Container Anti-Pattern are always a little bit of a wild ghoose chase, because thats based on the individual opinions, not on facts, which organizations like the CNCF defined (at least not that i know of). When i google for it, i get instantly deliver small throw-away images, like the one i provide. But in a way you could say, the image i deliver doesnt resembles the "container" i started, because a lot changes. I give you that, 100% right and i would also say this in a Dockerfile review at my company and say in a way thats not Best-Practices followed. But they also do web-development, not gameserver-hosting.

What I would propose you and co (or me if you want the help!) do is:

• Make server installation part of the container build process (Dockerfile)
• Grab the version of the downloaded server using RCON (TODO, is this necessary/possible?)
• Extract that as part of your CI and label the container with that semver. And mark it latest as ya do.

Once all this is done, cold-start times will be HUGELY reduced, builds are 'hermetic', and, maybe most importantly, you'll be creating a history of all the main server versions. This could be useful for posterity.

I even thought of it to do this in a docker-multistage build, release a slim version and fat version, there are even 2 versions to solve this, using a builder and copy the gameserver files later to the final image or just build the slim image and at the next image level, inject code that is similar to the "server.sh fresh-install function". So mechanically there is a way ... the version number to get shouldnt be that hard, but that version number doesnt mean anything ... because Pocketpair released multiple patches under the same version number, numerous of times. Details later ....

---

One issue with bundling the server software is that it would dramatically increase the size of the container image, as it is about 2GB at rest.

It would be 2,1 GB of gameserverfiles ontop of that, before compressing the image back down to who knows how many MB/GB.

In addition, that would put the burden of updating Palworld to the latest version on the container developer. Sure, CI jobs could update the container periodically, but it would never be up to date as updating Palworld locally.

Yes, what if im late or have like 5 days no internet, what then? Sure automation can be done, but on what KPIs to orient?! Also what about "Feature-Releases" of this image, do i have to tag now 3 new features with 1 server version? That surely wont get confusing for everyone involved ^^.

Another issue is that it would tie container and Palworld updates together, making it more difficult for users to avoid updating the container if they wish to keep the same version, but use the latest version of Palworld.

Yes again, Feature-Upgrade is now a level with server-updates. Also Pocketpair did already patches to the game in front and server and still used the same version number, they didnt increase even the bugfix number, but you could see the updates and the releases on client and server over steamdb, als this would increase comlexity and intransparency.

You've got many points here yes.

---

I had now time to think about it a few days and consult a few of my friends who understand Docker and Development in certain deep ways, thoose are more ways then just use immutable hosting, or webhosting, but also pipelining, gameserver-hostng and blueprinting via Docker/Images. The feedback was really big and many many questions arrised like:

• Q: Are you even allowed by license/eula to do that?
  • Me: Im not sure about that and dont need the trouble of legal stuff to be honest
• Q: Why is immutable having the gameserver beeing in there?
  • Me: Immutable means not changing over time, your image does that, what chanages is the assets which are stored in a volume. Even system conisdered immutable need as an example a database and store updates, data, key-values in there, why is yours different?
• Q: So when you go the long road of archiving every server-version there is (dont think you can do that, you have to be there for every patch that drops, even shadow-patches, with server version), how does that help you or the community? Are they even able to use that? What about clients that need to connect to the server? Do they miss-match and generate conflicts? Can a client go to steam or xbox and say "i want version from 3 days ago"?
  • Me: Not sure how that helps the community, but it will bloat GHCR and Docker-Hub with data for a long time. Even if im there for all releases and do every shadow-patch as an release and ontop of that the Feature-Request too, this will be a nightmare to use for new-users. And no, Pocketpair doesnt use the -beta XYZ version feature for Steam and SteamCMD so you can pick and choose what client you want.
• Q: Say i want to help you and test images and features, now i have to download 7 images which are FAT not slim, with only your codechanges. Why do i need that unnecessary traffic on my "limited" or "slow" internet-line? I only changed a few characters in docker-compose and the game is still on my disk, whats the value here expect wasting space for every user and on every container registry?
• Q: Even if im using the latest version, dont i need to check for updates that came out? You might be late ....
  • Me: You got a point and IF yes, then your version changes still by using version pinning
• Q: Why is it not okay to download the server after? Imagine its 30GB, can you even host that on Dockerhub?
• Q: Doesnt immutable referes on what you want to achieve and not what software you using?
  • Me: Fair point, depends on the idea and the mindset and the mechanics you using also ...
• Q: If that user wants to archive all the gameserver versions, whats the value? When do you omit old versions? Whats the rentention-policy?
• Q: Does it make even sense to offer different versions? For Node, Php etc. you need this. But old gameserver-versions? Why would that make sense? Doint you need a client in the same version?
• Q: Is it the focus of you image you want to provide? If no, why dont fork and do it yourself?

As you can see that topic isnt easy to come by and sometimes even i dont have a answer for the questions that arrised, because if you ask me, my focus is doing a good image with server-tools around Palworld-Dedicated-Server. Im following the use-case that many others already provided to keep buildtimes and image-sizes slim and download once the server on the users side and not to overuse traffic, that others might not have. This follows more the KISS principle imho. While the solution might be easy or hard to solve, im asking myself all the time, "is that even needed?" and i always come back to that question. Without Pocketpair not changing their release-features and using the -beta feature for Steam and SteamCMD i dont really see a value in all of this.

To be honest im smarter then before i went on this road to evalution of this, but i dont think my opinion changed from before and many questions stay un-answered, because an "idea" isnt something flushed out like a plan. So there is always uncertainty.

Any input from you guys, about the questions or even adding more questions to the stack-pile?

[jammsen]

Seems like this FR is dead. Im inactivity-closing this, feel free to contact me again to reopen it again.