Document how to configure the docker daemon from Butane

[jlebon]

See https://docs.docker.com/engine/reference/commandline/dockerd. E.g. to pick a few common ones: changing the log driver, debug mode, and insecure registries.

Would be nice to use configuration files, but it doesn't support drop-ins, so isn't very friendly to FCOS.

The moby-engine package we ship uses /etc/sysconfig/docker as an environment file for the daemon systemd unit. It's clunky but probably the cleanest approach for now is to do similarly to what's described in https://github.com/coreos/fedora-coreos-tracker/issues/588 and add a systemd unit override to just redefine ExecStart so we can append arbitrary flags.

(Though we should also look into what are the upstream plans for supporting a config overlay paradigm.)

Acceptance Criteria:

• document how to change the log driver
• document how to add insecure registries
• (optional) document how to enable debug mode

[J0WI]

Here is an example for Ignition:

systemd:
  units:
    - name: docker.service
      dropins:
        - name: 10-debug.conf
          contents: |
            [Service]
            Environment=DOCKER_OPTS="--debug"
            ExecStart=
            ExecStart=/usr/bin/dockerd --host=fd:// --exec-opt native.cgroupdriver=systemd $OPTIONS $DOCKER_OPTS

[dghubble]

To add declarative examples, you can also just overwrite the /etc/sysconfig/docker file. Since Docker unfortunately doesn't support configuration drop-ins.

variant: fcos
version: 1.0.0
storage:
  files:
    - path: /etc/sysconfig/docker
      overwrite: true
      contents:
        inline: |
          # Modify these options if you want to change the way the docker daemon runs
          OPTIONS="--selinux-enabled \
            --log-driver=json-file \
            --live-restore \
            --default-ulimit nofile=65536:65536 \
            --init-path /usr/libexec/docker/docker-init \
            --userland-proxy-path /usr/libexec/docker/docker-proxy \
          "

[J0WI]

To add declarative examples, you can also just overwrite the /etc/sysconfig/docker file.

This works, but I don't want to check it for changes from CoreOS itself.

[dghubble]

Well, both the service unit and the config cound be changed in a future update to docker. But figured I'd mention

[remoe]

Current FCOS use "--log-driver=journald" as default.

[robbertkl]

I'm new to FCOS, doing some demo deployments where I need custom Docker configuration.

For me, using Ignition to create /etc/docker/daemon.json works perfectly. No need to even create a systemd drop-in to override ExecStart (to pass --config-file to dockerd), as the config file is picked up automatically in this location.

It even merges the config with the arguments passed to dockerd by the systemd service, so you won't have to worry about the service unit or the sysconfig in a future release, as these stay untouched.

Any downsides / reasons I should not be doing it this way?

[sambernet]

@robbertkl while this works for cases where you exclusively need to add extra options to the upstream defaults, it will fail with an error in case some option is defined both in the upstream /etc/sysconfig/docker and your added /etc/docker/daemon.json files.

I used your above approach to add stuff like default-address-pool to the daemon options - but once I wanted to expose the docker daemon on a TLS socket by adding the hosts option, it failed loudly:

Reading config file '/etc/rpm-ostreed.conf'
Jan 15 03:37:35 build1 dockerd[2290]: unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: hosts: (from flag: [fd://], from file: [tcp>Jan 15 03:37:35 build1 systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Jan 15 03:37:35 build1 systemd[1]: docker.service: Failed with result 'exit-code'.
Jan 15 03:37:35 build1 systemd[1]: Failed to start docker.service - Docker Application Container Engine.

So while that approach has the benefit of not having to manually check for upstream config changes (configuring /etc/sysconfig/docker could be overwriting options that were added upstream later on, without us noticing), it can also potentially break on updates.

Probably that wouldn't fail on FCOS because auto-update should roll back to previous rpm-ostree snapshot, but it would then at least block automatic updates until resolved.

I like fail-fast, but the bottom line is - as usual: "it depends" 😇

PS: See also this discussion on the fedora mailing-list

[robbertkl]

@sambernet Thanks for the thorough explanation! All clear, I'll keep using it for my setup, as indeed I only add extra options.

Would be nice to have a single, fool-proof (and indeed documented) way of configuring the Docker daemon. Especially since running Docker (or other container systems) is one of the main tasks of a FCOS server.