Feature: Minimal Docker/Compose Infrastructure

[michaeldelago]

Describe your feature request or suggestion in detail

Hey all!

I've been looking through the Docker build and the docker-compose.yaml, and it seems like the default Docker builds are geared around development rather than deployment. I think using containers for development is fantastic, but I also think the approach recommended in the Docker setup guide should create an environment that's understandable to people who may not be familiar with AzerothCore, but are familiar with running containerized services.

Container builds (in the standard recommended method, ./acore.sh docker build) result in a few quite large containers:

$ docker image ls --filter "reference=acore/*:master"
REPOSITORY                         TAG       IMAGE ID       CREATED      SIZE
acore/ac-wotlk-worldserver-local   master    ef7d43e00b0d   5 days ago   1.58GB
acore/ac-wotlk-authserver-local    master    ea120a17973c   5 days ago   1.58GB
acore/ac-wotlk-dev-server          master    bc00ce91a1c3   5 days ago   3.17GB

It's fair to note that container builds are layered, but I personally think that layering is a "nice to have" rather than something that should be used as a main point; some deployment environments (Namely those where the authserver and the worldserver could end up on separate servers) won't be able to benefit from layering.

Going through the Dockerfile itself (as well as the docker-compose.yaml), a few things stand out to me that may be causing bloating the containers:

• All of ./data/sql is included in the container build
  • Increases container size by ~700MB
  • /data/sql is only used for a one time initialization job
• a Deno binary is included in the container build
  • Increases container size by ~100MB
  • All build/run operations for the container can be performed without using the acore.sh dashboard
• The servicebase stage includes a large amount of unneeded packages
  • Increases container size by ~600MB
  • It appears that gdb is installed, along with the -dev versions of many of the library dependencies

Other things I noticed in the Docker build that could be removed to reduce complexity:

• simplify the compilation steps
  • many short shell scripts with includes that source other short shell scripts with their own includes, making it difficult at times to trace what exactly is happening
• mounting a docker volume containing the compiled binary at runtime

Describe a possible solution to your feature or suggestion in detail

I think a new Dockerfile and docker-compose.yaml should be written, in order to accommodate a more minimalist docker build. Goals would include:

• Remove large sources of disk usage
  • Don't include ./data/sql in the final build
  • Don't include Deno in the final build
  • Ensure all packages in the final build are required, and that it's using the smallest versions of those
• All data and configuration should be attached through volumes on the containers. The containers should not contain data that changes or isn't accessed during runtime.
• Simplify execution by starting the binary outright, or having a thin shell script to wrap the execution of the binary
• Modules should still work in the standard way
• The DB Import should be a part of the docker-compose stack, and a prereq for the worldserver
• Retrieving Client Data should be a part of the docker-compose stack, and a prereq for the world-server
  • Note, this will require a ./env/docker/data/.initialized file created to prevent downloading the client data more often than is needed
• Don't break the previous docker setup.
  • ./acore.sh docker ... is valuable to development, and should still work.

Secondary goals:

• Throw a simple "Quick start" into the readme that essentially says:
  1. Download a WoW 3.3.5a client, set realmlist
  2. Clone this repo
  3. Execute docker-compose up --build
  4. Play WoW
  5. For more information or advanced setup, check out ./acore.sh --help or the wiki
• Creation of an admin account on startup (again with some sort of .initialized file or another way to ensure it only happens once)

Additional context

I don't intend to come across as overly-critical - it's plainly obvious that a lot of time, effort, and love has gone into the development of this corner of the project. That said, I feel like there's some parts that can be improved, and one of those is something I have experience in.

From a development standpoint, I think the docker builds are fantastic, but I don't think the (likely to be very common) use case of someone stumbling on the project and wanting to spin up an idiomatic docker-compose stack is being properly met.

I'm aware of acore-docker, though I believe that repository is in a similar boat where it seems to have too much of a focus on developing Azerothcore and less on concisely describing a standard way of deploying Azerothcore. If anything, I feel like it wouldn't be terrible archive that repo, and have docker-based deployment described in this one.

I'm new to AzerothCore development, so if it seems like I'm missing something, I probably am. Please let me know!

Edit Jan 14, 23: I forgot to mention - I've begun working on the proposed changes as of earlier in the week. Currently getting the authserver to start with a ~180MB container build.

[michaeldelago]

It's a start, I guess

$ docker image ls --filter "reference=acore/*:docker-multi-stage"
REPOSITORY                   TAG                  IMAGE ID       CREATED          SIZE
acore/ac-wotlk-tools         docker-multi-stage   fa5bbb946c35   14 minutes ago   189MB
acore/ac-wotlk-authserver    docker-multi-stage   5c1df1171a49   14 minutes ago   168MB
acore/ac-wotlk-worldserver   docker-multi-stage   7bb0f8dbe993   14 minutes ago   769MB
acore/ac-wotlk-db-import     docker-multi-stage   3ae4da780017   14 minutes ago   160MB
acore/ac-wotlk-client-data   docker-multi-stage   30b1bc69f7c5   15 minutes ago   93.3MB

Whole set of images totals out to 1379MB

$ docker image ls --filter "reference=acore/*:docker-multi-stage" --format '{{ .Size }}' | \
    awk '{s+=$1} END {print s}'
1379.3

Compared to images created by standard build in master

$ docker image ls --filter "reference=acore/*:master"
REPOSITORY                         TAG       IMAGE ID       CREATED      SIZE
acore/ac-wotlk-worldserver-local   master    a8da0781742d   2 days ago   1.49GB
acore/ac-wotlk-authserver-local    master    71afd9658085   2 days ago   1.49GB
acore/ac-wotlk-dev-server          master    9f49a08584eb   2 days ago   3.71GB
...
$ docker image ls --filter "reference=acore/*:master"  --format '{{ .Size }}' | \
    awk '{s+=$1} END {print s}'
6.69 # GB

I haven't tested these initial images yet, I expect there to be changes needed that make them larger

[michaeldelago]

Hitlist and things that I think would help simplify the docker install. Please feel free to note what you'd prefer to be different

Note that this is mostly my own goals/interest for this and don't necessarily reflect AzerothCore as a whole

• [x] Simplify configuration for services in containers (authserver, worldserver, dbimport)
  • resolved by https://github.com/azerothcore/azerothcore-wotlk/pull/16817, https://github.com/azerothcore/azerothcore-wotlk/pull/17019 and https://github.com/azerothcore/azerothcore-wotlk/pull/17040
• [x] feat(docker): Implement self-contained docker build
  • The "standard" docker build (as in, ./acore.sh docker build) builds images by first building a container with all the dependencies (called the dev-server) and then in that container building the binaries for the server. After this, separate worldserver and authserver containers mount those binaries.
  • This is complex and doesn't allow docker to use cache properly, leading to situations where cache has to be cleared frequently in order to prevent the build from failing. Additionally, this causes disk usage on the host machine to grow to hundreds of GB because again, cache is not managed properly.
• [x] feat(docker): remove dev/prod distinction in docker images
  • There's a distinction in the docker builds for "dev" and "prod"
  • the only difference is that prod images have eluna built in and they compile their binaries as a part of the build
  • the acore-docker repo uses the "prod" images, but it's a pretty simple lift to pull in eluna during the CI process that publishes the containers for acore-docker
• [x] feat(docker): remove dependence on deno
  • Deno doesn't support arm64 MacOS builds
  • Deno is used mainly for docker, where it is called by shell and then executes shell commands
  • All the things deno does can just be done in shell and Deno's use here bloats up the install.
  • Addressed in https://github.com/azerothcore/azerothcore-wotlk/pull/16934 and https://github.com/azerothcore/azerothcore-wotlk/pull/16898 (considerable overlap between the 2. The first one should be picked, in my opinion. The second on keeps the existence of deno but removes it from the docker build)
• [x] feat(CI/Docker): Simplify CI processes
  • The CI for building/pushing the docker containers is very large and again doesn't use cache well.
• [ ] feat(CI): We should set up automatic, date based tagging in github
  • The closest thing we have to "releases" is the database being squashed
  • This creates a new folder in https://github.com/azerothcore/azerothcore-wotlk/tree/master/data/sql/archive/db_world with a version
  • After a database squash, you need to pull the "last" commit for that version and execute all database migrations before running the new version
  • Putting these versions into tags (every day at 12PM UTC is honestly probably fine) is a super simple solution to the problem to specify which commit you need
  • proposed version schema "vX+.YYMMDD" where X is the DB version (taking as many spaces as needed), YY is year, MM is month, DD is day (all date params taking 2 spaces each)
• [ ] feat(Core): Allow full administration without the console/captive shell
  • The interactive console is a bit of an oddity. It's not common to docker attach to a container as part of the initialization process.
  • It's good that it's a one-time thing if you're just running a server, but it's something that becomes clunky quite quickly if you're trying to test a PR with docker
  • All it really needs is to be able to set the password of a user account in the config, similar to how it works for the mysql container.

Vaporware: Things to explore after we reinforce the base

• A good "getting started" section in the Readme
  • I personally don't like the "Go to our wiki to figure out how to set this up"
  • A "quick start" having the basic process to compile and jump into the game would probably be a good idea and a better user experience
  • I don't intend for this to be docker specific, but docker is the least painless way to do this
  • At the end of the quick start, we can note that further documentation is in the Wiki
• Simplify build processes
  • A lot of the acore.sh dashboard jumps around throughout the repo.
  • Many shell scripts get sourced and overall there's a lot of "magic" that makes the build process more complex than it needs to be or should be.
• Kubernetes support (@walkline is exploring this with https://github.com/azerothcore/azerothcore-wotlk/discussions/16748 and https://github.com/walkline/ToCloud9/tree/master)
  • most likely, a helm chart would be "contrib" (as in best-effort support)
• Modules as shared libraries
  • This opens the door for making module installation as easy as downloading the .so or .dll from github
  • I'm poking at this myself, it looks like the ScriptReloadMgr code (https://github.com/trinitycore/TrinityCore/blob/3.3.5/src/server/game/Scripting/ScriptReloadMgr.cpp) from trinitycore can be ported
  • I'm not particularly interested in hot reloading of modules but that's a possibility.

[walkline]

feat(Core): Allow full administration without the console/captive shell

Do you have any ideas how we can achieve this?