[BUG] docker compose alpha watch does not recreate container with current config if config changes while command is running.

[max-cole]

Description

When using the docker compose alpha watch command if you make a change to the env_files in the docker-compose.yaml or change the docker-compose.yaml file itself and let a file change trigger the rebuild functionality those changes are not used during the rebuild.

Steps To Reproduce

Working Docker Desktop

Use the following Config.

docker-compose.yaml

services:
  web:
    build: .
    command: npm start
    env_file:
      - ../web.env
    x-develop:
      watch:
        - action: rebuild
          path: package.json

web.env

VALUE=1

If you do a docker compose up -d the web container will be created with VALUE=1 as an env-var.

You can then start a docker compose alpha watch to make your changes.

If you change VALUE=1 to VALUE=2 in the .env file this will not trigger a rebuild (as expected we are not watching that file) and if you do a docker-compose up -d the container will be recreated with the new value. However if you don't stop the watch and it triggers a rebuild the container will be recreated with the value at the time the docker compose alpha watch was started.

This also works if you have the watch include the .env file, the container will be recreated with whatever was in the .env file at the time of the docker compose alpha watch command was started.

If you change the config of the docker-compose.yaml file this also does not get carried over when the rebuild starts.

docker-compose.yaml

services:
  web:
    build: .
    command: npm start
    env_file:
      - ../web.env
    environment:
      - name=value
    x-develop:
      watch:
        - action: rebuild
          path: package.json

Triggering a rebuild via the watch functionality creates a new container without the name=value env-var unless you also restart the watch command.

Expected Behavior:

The watch command should rebuild the full config of the container including any changes to configs whenever a rebuild is triggered.

When I make a change to the env file I do not expect the rebuild to trigger if I'm not including that in my watch config, however when that rebuild does happen I would expect that the most current config would be used to rebuild the container not the config that was in place when the docker compose alpha watch command was started.

Compose Version

docker compose version
Docker Compose version v2.20.2-desktop.1

Docker Environment

docker info
Client:
 Version:    24.0.5
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2-desktop.1
    Path:     /Users/user/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.20.2-desktop.1
    Path:     /Users/user/.docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/user/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.20
    Path:     /Users/user/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.6
    Path:     /Users/user/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/user/.docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /Users/user/.docker/cli-plugins/docker-scan
  scout: Command line tool for Docker Scout (Docker Inc.)
    Version:  0.20.0
    Path:     /Users/user/.docker/cli-plugins/docker-scout

Server:
 Containers: 12
  Running: 11
  Paused: 0
  Stopped: 1
 Images: 22
 Server Version: 24.0.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
 runc version: v1.1.7-0-g860f061
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 5.15.49-linuxkit-pr
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 7.765GiB
 Name: docker-desktop
 ID: 23d03acd-9f09-4984-86b9-3d25b66c3fac
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

No response

[ndeloof]

watch indeed takes place after the compose file and related files have been parsed. It will neither re-create container with an updated config if you change anything in yaml. That's probably confusing, but with current architecture I don't expect this to be feasible. cc @milas

[g0t4]

How about an action type of recreate to support the following?

  # ...
  watch:
    - action: recreate
      path: compose.yaml
    - action: sync
      path: ./web
      target: /src/web
      ignore:
        - node_modules/
    - action: rebuild
      path: package.json

Then, if I edit actions or a port on a service definition, watch can trigger docker compose up -d or similar.

[max-cole]

That sounds great! I think that solution would work well for env files:

Ex in our local dev setup we have a folder with several env files for the services so its

~/local-dev/
~/local-dev/env/
~/local-dev/env/service1.env
~/local-dev/env/service2.env

I would love for a self-referential in the docker-compose file but that's a somewhat bigger ask something like:

services:
     service1:
     [...]
 watch:
     recreate-on-change: true

But the work around to simply put the compose.yaml into any service's watch section would work totally fine and be more in spirit with the rest of this work. Thanks!