[BUG] template_driver being ignored in config

jedahan commented 1 year ago

Description

I am trying to use secrets in a config template to work around zitadel not supporting file-based secrets management.

It seems the config is correctly identified and mounted, but not executing the golang template wrapper. I would expect the output to be Hello, 007 instead of Hello, {{ secret "codename" }}.

Otherwise, I would expect some sort of error message and early exit if template_driver is incorrect.

$ docker compose up
[+] Building 0.0s (0/0)                                                                   docker:orbstack
[+] Running 1/0
 ✔ Container compose-scratch-app-1  Recreated                                                        0.0s 
Attaching to compose-scratch-app-1
compose-scratch-app-1  | hello, {{ secret "codename" }}
compose-scratch-app-1 exited with code 0

compose.yaml

version: "3.8"

secrets:
  codename:
    file: codename

configs:
  greeting:
    file: greeting
    template_driver: golang

services:
  app:
    image: alpine
    secrets: [ codename ]
    configs: [ greeting ]
    command: [ "sh", "-c", "/greeting" ]

greeting

echo Hello, {{ secret "codename" }}

codename

Steps To Reproduce

Use the above files, then run docker compose up, and observe that the configs are not being run through the specified template_driver.

Compose Version

$ docker compose version
Docker Compose version v2.22.0

$ docker-compose version
Docker Compose version 2.23.0

Docker Environment

Client:
 Version:    24.0.6
 Context:    orbstack
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.1
    Path:     /Users/micro/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.22.0
    Path:     /Users/micro/.docker/cli-plugins/docker-compose

Server:
 Containers: 4
  Running: 0
  Paused: 0
  Stopped: 4
 Images: 8
 Server Version: 24.0.6
 Storage Driver: overlay2
  Backing Filesystem: btrfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 091922f03c2762540fd057fba91260237ff86acb
 runc version: 82f18fe0e44a59034f3e1f45e475fa5636e539aa
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.5.7-orbstack-00109-gd8500ae6683d
 Operating System: OrbStack
 OSType: linux
 Architecture: aarch64
 CPUs: 10
 Total Memory: 7.748GiB
 Name: orbstack
 ID: 2a4af27e-188d-4bd2-9a74-02baef220b6b
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine
 Default Address Pools:
   Base: 192.168.215.0/24, Size: 24
   Base: 192.168.228.0/24, Size: 24
   Base: 192.168.247.0/24, Size: 24
   Base: 192.168.207.0/24, Size: 24
   Base: 192.168.167.0/24, Size: 24
   Base: 192.168.107.0/24, Size: 24
   Base: 192.168.237.0/24, Size: 24
   Base: 192.168.148.0/24, Size: 24
   Base: 192.168.214.0/24, Size: 24
   Base: 192.168.165.0/24, Size: 24
   Base: 192.168.227.0/24, Size: 24
   Base: 192.168.181.0/24, Size: 24
   Base: 192.168.158.0/24, Size: 24
   Base: 192.168.117.0/24, Size: 24
   Base: 192.168.155.0/24, Size: 24
   Base: 192.168.147.0/24, Size: 24
   Base: 192.168.229.0/24, Size: 24
   Base: 192.168.183.0/24, Size: 24
   Base: 192.168.156.0/24, Size: 24
   Base: 192.168.97.0/24, Size: 24
   Base: 192.168.171.0/24, Size: 24
   Base: 192.168.186.0/24, Size: 24
   Base: 192.168.216.0/24, Size: 24
   Base: 192.168.242.0/24, Size: 24
   Base: 192.168.166.0/24, Size: 24
   Base: 192.168.239.0/24, Size: 24
   Base: 192.168.223.0/24, Size: 24
   Base: 192.168.164.0/24, Size: 24
   Base: 192.168.163.0/24, Size: 24
   Base: 192.168.172.0/24, Size: 24
   Base: 172.17.0.0/16, Size: 16
   Base: 172.18.0.0/16, Size: 16
   Base: 172.19.0.0/16, Size: 16
   Base: 172.20.0.0/14, Size: 16
   Base: 172.24.0.0/14, Size: 16
   Base: 172.28.0.0/14, Size: 16

Anything else?

I linked to the wider context/motivation re: zitadel in the original description - if there is a more correct usage of docker features than configs+secrets, I'd love to learn them.

ndeloof commented 1 year ago

As noted on https://docs.docker.com/compose/compose-file/compose-file-v3/#configs-configuration-reference:

only supported when using docker stack.

Docker Compose has no "extensibility" mechanism that would allow to run such a driver (client-side, as compose has no engine component). The closer solution I have in mind for your use-case is https://github.com/compose-spec/compose-spec/pull/429 (not released yet)

ndeloof commented 1 year ago

keeping issue opened, as compose should reject a compose file using unsupported attribute

jedahan commented 1 year ago

@ndeloof thank you for both the immediate fix in #11185 and the long-term solution in compose-spec/compose-spec#429

If I want to be informed of when that feature is released, is there an issue/PR/repository/blog I can watch or subscribe to? Or is there a way to test that feature today (using a nightly compose?)

ndeloof commented 1 year ago

this will be part of the next docker compose release

docker / compose