[BUG] docker compose config merges environment variables into a list

jcsawyer commented 2 weeks ago

Description

When using docker compose config with multiple json files, the environment variables are being merged into a flat list of strings where previously this was a dictionary.

Expected behaviour (tested from) Docker Compose version v2.24.6-desktop.1:

Name                           Value
----                           -----
environment                    {[someVariable, test]}

Actual behaviour in Docker Compose version v2.27.0-desktop.2:

Name                           Value
----                           -----
environment                    {someVariable=test}

Steps To Reproduce

Have a base compose file docker-compose.base.json:

{
"services": {
"serviceA": {
  "image": "serviceA"
},
"serviceB": {
  "environment": {
    "someVariable": "test"
  },
  "image": "serviceB"
}
}
}

Have an overrides compose file docker-compose.overrides.json:

{
"services": {
"serviceB": {
  "environment": {
    "variableOverride": "test"
  },
  "image": "serviceB:build"
}
}
}

Run compose config:

$config = docker compose -f docker-compose.base.json -f docker-compose.overrides.json config --no-interpolate --format json | ConvertFrom-Json -AsHashtable

The environment variables become a list of strings:

> $config.services.serviceB.environment
variableOverride=test

Compose Version

PS C:\> docker compose version
Docker Compose version v2.27.0-desktop.2
PS C:\> docker-compose version
Docker Compose version v2.27.0-desktop.2

Docker Environment

PS C:\> docker info
Client:
 Version:    26.1.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.14.0-desktop.1
    Path:     C:\Program Files\Docker\cli-plugins\docker-buildx.exe
  compose: Docker Compose (Docker Inc.)
    Version:  v2.27.0-desktop.2
    Path:     C:\Program Files\Docker\cli-plugins\docker-compose.exe
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.29
    Path:     C:\Program Files\Docker\cli-plugins\docker-debug.exe
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     C:\Program Files\Docker\cli-plugins\docker-dev.exe
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.23
    Path:     C:\Program Files\Docker\cli-plugins\docker-extension.exe
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.4
    Path:     C:\Program Files\Docker\cli-plugins\docker-feedback.exe
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.1.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-init.exe
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-sbom.exe
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-scan.exe
  scout: Docker Scout (Docker Inc.)
    Version:  v1.8.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-scout.exe

Server:
 Containers: 56
  Running: 1
  Paused: 0
  Stopped: 55
 Images: 363
 Server Version: 26.1.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e377cd56a71523140ca6ae87e30244719194a521
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
 Kernel Version: 5.15.146.1-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 15.49GiB
 Name: docker-desktop
 ID: effb060b-7e0b-42d0-9870-fa27f90883af
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=npipe://\\.\pipe\docker_cli
 Experimental: true
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

This also affects YAML.

ndeloof commented 2 weeks ago

Both are valid and equivalent in compose file format according to the compose specification. Support for --no-interpolate requires we use the key=value syntax as some of those maybe use a variable and can't be split into a key: value mapping.

Also, uncommon to declare compose file using json, any reason you adopted this syntax ?

jcsawyer commented 2 weeks ago

We were using this functionality in our CI builds to deploy an Azure container instance for Windows container services by token replacing the values and leveraging the key value pair to build a bicep deployment parameters file.

Our compose files are defined as YAML but we build JSON files in pipelines as it sped up our builds.

ndeloof commented 2 weeks ago

If you have a custom compose file parser, then this one should follow the compose specification and support the multiple syntaxes.

docker / compose