Securely manage traefik credentials

[bmschmidt]

Low priority--find a way to get @senderle's personal traefik ssl credentials out of the default build and move into a configuration file somehow.

(I say low priority on the assumption that this will at least work over port 80 without it, which I don't know to be the case).

[senderle]

Yep, this is an annoying aspect of traefik's configuration scheme. It affects the domain name as well. Options:

• Template the yaml and do a substitution during the image build phase, based perhaps on environment variables?
• Use the environment variable config scheme. Except traefik doesn't let you mix them; it's all yaml or all env. variables, and the number of env variables we'd need is annoyingly large.
• Traefik has a concept of a config "provider." The config file is one kind of provider but there are other providers, including I guess docker itself? I don't understand this part of traefik well but we could look into it. It seems like it might be a more robust solution in the long run.
• Just drop SSL. I would still want to use traefik since it makes it easy to add SSL later. If we drop SSL, neither domain name nor email is necessary, and a flat config file is fine. This might be the best option for now.

[bmschmidt]

The key question for me here is--will port 80 still work with all this stuff in there? Not worth breaking http for the sake of https in the standard container--especially since there are many other ways to handle https--but it would be nice to provide a decent https skeleton.

[senderle]

The standard approach is to silently redirect to https on a http call, and that has always worked smoothly for me. But the more we talk about this, the more I think we should just get it working with vanilla http first, and reckon with these details later. As long as we are using traefik as a reverse proxy, we can defer this problem.

[senderle]

Dropping a couple of resources here for future work on this. These demonstrate how to use the docker provider with traefik & let's encrypt in a relatively clean way. Still a little verbose, but would mean stuff like domain name & email aren't buried in a hard-to-find yaml file. Also seems like this could work well with the .override approach that @bmschmidt wants to try. Edit: Nope. At least not yet.

https://www.google.com/amp/s/www.simplecto.com/traefik-2-0-docker-and-letsencrypt/amp/

https://doc.traefik.io/traefik/user-guides/docker-compose/acme-tls/

[senderle]

~Using the docker provider turned out not to be so hard. The current code in the docker-provider branch works with a dummy flask app in place of the API (which still needs work). You have to override five sections, and right now, where things are inside lists, you have to override the whole list. Which is not ideal — I have to see if we can make overriding work for individual list items.~

Everything above turns out to be false because I had an old traefik image build that was still using the file provider. This branch doesn't work with the override file at all. I am not sure why. It does work if you put everything directly in docker-compose.yml. Which is less ideal, though perhaps still tolerable.

Also apparently it has to be docker-compose.override.yml — note the substitution -override -> .override.

[bmschmidt]

Not sure if this is the issue, but one problem with docker-compose.override.yml is that it doesn't overwrite existing keys, just adds new ones. So if you have

...
    ports:
      - "0.0.0.0:80:80"
      - "0.0.0.0:443:443"

And then write into override

...
    ports:
      - "0.0.0.0:8080:80"
      - "0.0.0.0:443:443"

It still tries to serve over 80 at the same time. (This makes the whole thing fail on my local machine, where I'm already using port 80 for something else; and where I don't think my userland docker image should be allowed to commandeer port 80, anyway.

Since it's perfectly reasonable to want to run a bookworm instance locally on port 8080, this suggests to me some need for a general rethink. Gonna make a different comment to that effect.