Containerize application with Docker

[tralph3]

I have added a Dockerfile that builds the client from source and runs the server. I saw in other issues that you are still learning about Docker, so I'll explain how it works so it's easier for you to follow. Excuse me if I give too much detail, but I don't know how much you know about Docker, so I'll assume you know very little.

In Docker, you first build images. Images are sort of blueprints. They are static, they can't be used as-is, and they contain all the files that would be stored inside a Docker container. From these images, you create the containers themselves. The Dockerfile contains the instructions to make these images.

Once an image is built, you can create as many containers from it as you wish. These containers can then be modified. Their internal filesystem can be changed, and you can run applications within them. However, since they are separated from the rest of your system, once the container is deleted, anything inside it is lost. In order to persist data, you need to use bind mounts, or volumes.

Bind mounts essentially map a folder in your regular system, to a folder inside the container. Whatever data gets written into the folder, is reflected in both systems. You can then delete and re-create containers as you see fit, and they will pick-up this bind mount and can read their files. A very common thing to use them for is config files. Programs should store all the configurations, databases, or other persistent data in a single folder, which can then be bind mounted. Later, if you update the application, you remove the container, re-create it from an updated image, and it can re-read the bind mount to restore the configuration.

Volumes work similarly, but they are managed by Docker. They also kind of map a folder in your filesystem to one in your container, but you can't choose what the folder in your filesystem is. It's stored in a special directory intended for Docker alone, and they are used for data that you want to persist, but shouldn't be directly handled by the user.

This is something the user chooses how to use tho, the image only states what folder or folders should be binded, but doesn't enforce it.

So, with that out of the way, I'll proceed to explain how this particular Dockerfile works.

You can see that it starts with a FROM directive. This means that the Docker image we want to build, will use another, separate image as its base. Since we don't want to install all the dependencies that make the entire operating system, we use one of these base images.

In this case, since your program is split into a client and server, I used a multi-stage build process. I temporarily use the node image to build the client, and that gets copied later on to the actual final image.

You can see I run a couple commands with the RUN directive, you can guess what they do. WORKDIR essentially sets the current directory, like using cd on a shell.

We then encounter a second FROM directive. This is the base image the final image will be based on (that's a mouthful). We set a WORKDIR like before, and we COPY everything from the current directory (that would be this repository) into the current directory inside the container (set by WORKDIR).

We then also COPY from the previous image, which was labeled as CLIENT in the very first line. The first path corresponds to the CLIENT image, and the second path corresponds to the current (final) image, and is relative to WORKDIR.

We then expose the port the program uses. This is the same concept as opening ports on a router. Docker has its own internal networks for each container after all.

I also set some VOLUMEs. Remember, the user should actually bind folder or volumes to these folders. If they don't, docker creates dummy volumes for them, but this actually works more as documentation than anything else. Of course, these folders aren't actually checked by your program. I couldn't find where it stores config files, or even if they are centralized in a single folder or file. If they don't, ensure they do, and change that line to point to it (remember that the repo is stored in /app/swingmusic in the container).

After that I install poetry and the needed dependencies. I disable the virtualenvs option for poetry, since this is running in a container, it's already isolated.

Finally, the CMD directive is the command to run when the container starts. I also set the host to 0.0.0.0 so it is broadcasted to the local network, needed if you want to be able to access it from outside the docker container.

So now, if you want to try it out, clone the repository with the Dockerfile, and run:

docker build . -t swingmusic

If your user is not part of the docker group, then you will need to use sudo. This assumes you're on Linux of course. If you're not, then I'm afraid you'll have to look somewhere else :P

Anyway, the command will look for a file named Dockerfile by default, and will build it. The -t swingmusic part sets a tag for the final image, basically a name. It's optional, but setting it makes it easier to work with later on.

Once the image is created, you can see it with:

docker image ls

There's likely more than one, since more images had to be downloaded to build this one.

You can then create a container from this image like this:

docker run --name swingmusic -p 1970:1970 swingmusic

This will create a container named swinmusic, it will bind the port 1970 in your machine, to the port 1970 inside the container. Without this, you wouldn't be able to access the site. And lastly, at the very end, we specify the name (or tag) of the image we want to use to create this container, swingmusic in this case.

I have noticed that after doing this, the terminal gets taken over by the program's output. You should try to fix that, since it's not expected for a program to do that.

To read the program output for a docker container, you can use:

docker logs swingmusic

Or the name of any other container you want.

To remove the container and the image, first stop it, then remove them both:

docker stop swingmusic
docker rm swingmusic # this removes the container
docker image rm swingmusic # and this the image

Do ask if you have any other questions. You can then see if you can submit this to Dockerhub, or Quay, or some other public docker image repository.

[tralph3]

Oh, btw, if you want to inspect the files inside a docker container, so that you can see how the final image is organized, you can do:

docker exec -it swingmusic bash

This will open a bash shell and you can use all the coreutils you know and love to navigate through it.

[cwilvx]

Hello @tralph3

Thank you very much for this. The explanation is very nice 👍. I have gone from knowing nothing about Docker to actually running Swingmusic in a docker container with this PR 😁.

[cwilvx]

If you don't mind, would you please add instructions (like about volumes and other stuff ) in the readme for other Docker users? Your help is highly appreciated.

[cwilvx]

@tralph3

I noticed that symlinks are breaking for some reason. I used the -v flag to mount a folder to /music. Is there a way to mount multiple dirs to the same folder?

[tralph3]

@tralph3

I noticed that symlinks are breaking for some reason. I used the -v flag to mount a folder to /music. Is there a way to mount multiple dirs to the same folder?

What symlinks? I don't think you can bind multiple folders to the same folder in the container.

[cwilvx]

@tralph3 I noticed that symlinks are breaking for some reason. I used the -v flag to mount a folder to /music. Is there a way to mount multiple dirs to the same folder?

What symlinks? I don't think you can bind multiple folders to the same folder in the container.

That's okay. We'll find a way to handle multiple directories. Whatever we have will do for now.

[tralph3]

I mean, you can have a single directory /music inside the container for example, and then bind that folder somewhere in your system. Then, in your system, you can have subfolders there, and they will be reflected in the container as well.

However as I said, the VOLUME directive in the Dockerfile serves more as documentation than anything else. The user can bind any folder they want to any other, so they can drop the music wherever they want.

You DO need to worry about the config tho. Make sure to store all the config files or databases you use in a single directory, ideally at the root of the container (like /config) so that users can easily bind volumes or folders there. Currently you're storing them in the user's config folder, which is fine when running on bare metal, but on docker it's kind of weird to do that.

[cwilvx]

Aaaahh 😃 .... now I get it.

You can mount any number of dirs with the -v flag. I've tested with a few and it works as I expected.

So, for the config folder, I need to allow the user to set the config directory using something like this:

swingmusic --configpath /home/user/somedir

Is that correct?

[tralph3]

I added some instructions to the README.

[tralph3]

Aaaahh smiley .... now I get it.

You can mount any number of dirs with the -v flag. I've tested with a few and it works as I expected.

So, for the config folder, I need to allow the user to set the config directory using something like this:

swingmusic --configpath /home/user/somedir

Is that correct?

Yes, although this won't be possible to do with the current Dockerfile. I can modify it to allow for such a thing.

The problem being: I'm using the CMD directive by itself.

You can override the CMD directive when you create a container by just passing any extra argument you want at the end of the docker run command like this:

docker run swingmusic bash

This would run bash instead of the command defined in CMD (which is poetry run python main.py). What we can do, is define an ENTRYPOINT instead.

The entrypoint serves as the base command, and allows you to override the CMD directive without screwing up the intialization of the server.

Basically, we can set an ENTRYPOINT like this:

ENTRYPOINT ["poetry", "run", "python", "manage.py"]

And then define a CMD:

CMD ["--host", "0.0.0.0"]

The CMD will get appended to the entrypoint to form the following:

poetry run python manage.py --host 0.0.0.0

But since the CMD can be overriden, now the user gets the ability to pass any flags they want:

docker run swingmusic --config /config

This would replace CMD to --config /config. Problem is, now the program wouldn't work because the user has overriden the --host parameter so it won't be exposed on the LAN. But that's a problem the user should solve, since they should know what they're doing.

Or, we can also include the --host and the --config parameters as part of the entrypoint. That way, they will always be present, although you should override their values if the user manually passes these arguments.

I think it'd be great if we set the config folder location to /config in the entrypoint.

I'll modify the Dockerfile to include this.

[cwilvx]

Thank you for the README instructions.

I agree with the host and config flag being part of the entry point and we can set the config path to default to /config. I really like how you know your stuff (Docker).

In the meantime, let me go and put together that --config flag ... because with the current code, it's not recognized.

[cwilvx]

@tralph3 I added support for the --config flag. feel free to pull the changes for testing.

I'll go through everything one last time (the code and Docker stuff) later then merge this PR. Thank you for the contribution. Highly appreciated.

[tralph3]

No problem. I have actually already tested the flag yesterday, and it seems to be working. I saw there were some assets there? Maybe not the best place to drop them. But, that's something that can be tweaked later.

Feel free to merge.

[tralph3]

Maybe you should look into uploading it to Dockerhub or Quay, that way people can install the application without having to download the repo or anything, they just download the image from Dockerhub directly.

[cwilvx]

Sure. I'll look into it.

[cwilvx]

No problem. I have actually already tested the flag yesterday, and it seems to be working. I saw there were some assets there? Maybe not the best place to drop them. But, that's something that can be tweaked later.

Feel free to merge.

@tralph3 -- I actually missed this comment. What would you suggest about the assets?

[tralph3]

Remember that the config directory is meant to store only things related to configuration. If you reinstall the program from scratch (which it's essentially what creating a new docker container does), then these config files are the only thing that you should need to restore the previous state of the program. Here you should also store any database you use to keep track of the music files and such.

Assets should not be there, since they're not part of the configuration. They are expected to be updated by the program itself, or handled by the program itself, the user should not fiddle with the assets. As such, they should be stored somewhere else. Doesn't matter where, just not in the config folder. Unless I'm mistaken as to what those actually do.