x-develop/watch - Watching changes in additional_contexts

[tr00st]

Description

Somewhat related to comments made re: watch.source paths being relative to the build.context (see https://github.com/compose-spec/compose-spec/pull/253 )

Assuming it's intended long-term that build context is the basis for a x-develop/watch path, it would be useful to specify a reference to build.additional_contexts to allow syncing of code outside the main build context. Something like a from_context entry (named based on COPY --from= syntax from the Dockerfile, though "watch from" probably makes less sense semantically). Referencing paths outside build_context would also work for this use case, but I'm assuming there's some implementation detail that'd make that a bit more difficult.

For our specific use case, with have a lot of smaller repositories, which are consistently checked out - though the same could be applied to a monorepo as per the example I've dropped in below. Each app/product has a repo, and modules (referenced as a dependency in package.json to keep things separate). In an ideal world, any change made in a module should propagate an update to all dependent containers. We've used bind-mounts to achieve this with limited success, though performance is pretty poor (likely due to most of us running Windows).

I've thrown together a dumbed-down repo that demonstrates the sort of format I'm talking about, because code probably speaks louder than words. In reality, we use yarn link to reference the modules from our app repos, but the same principles should apply here.

https://github.com/tr00st/docker-compose-watch-additional-contexts

• edit - Adding docker version/context/info as requested on the issues intro:

  
  Client:
  Cloud integration: v1.0.33
  Version:           24.0.2
  API version:       1.43
  Go version:        go1.20.4
  Git commit:        cb74dfc
  Built:             Thu May 25 21:53:15 2023
  OS/Arch:           windows/amd64
  Context:           default

Server: Docker Desktop 4.20.0 (109717) Engine: Version: 24.0.2 API version: 1.43 (minimum version 1.12) Go version: go1.20.4 Git commit: 659604f Built: Thu May 25 21:52:17 2023 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.6.21 GitCommit: 3dce8eb055cbb6872793272b4f20ed16117344f8 runc: Version: 1.1.7 GitCommit: v1.1.7-0-g860f061 docker-init: Version: 0.19.0 GitCommit: de40ad0 PS C:\Work\Code\docker-compose-watch-additional-contexts> docker context show default PS C:\Work\Code\docker-compose-watch-additional-contexts> docker info Client: Version: 24.0.2 Context: default Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc.) Version: v0.10.5 Path: C:\Program Files\Docker\cli-plugins\docker-buildx.exe compose: Docker Compose (Docker Inc.) Version: v2.18.1 Path: C:\Program Files\Docker\cli-plugins\docker-compose.exe dev: Docker Dev Environments (Docker Inc.) Version: v0.1.0 Path: C:\Program Files\Docker\cli-plugins\docker-dev.exe extension: Manages Docker extensions (Docker Inc.) Version: v0.2.19 Path: C:\Program Files\Docker\cli-plugins\docker-extension.exe init: Creates Docker-related starter files for your project (Docker Inc.) Version: v0.1.0-beta.4 Path: C:\Program Files\Docker\cli-plugins\docker-init.exe sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.) Version: 0.6.0 Path: C:\Program Files\Docker\cli-plugins\docker-sbom.exe scan: Docker Scan (Docker Inc.) Version: v0.26.0 Path: C:\Program Files\Docker\cli-plugins\docker-scan.exe scout: Command line tool for Docker Scout (Docker Inc.) Version: v0.12.0 Path: C:\Program Files\Docker\cli-plugins\docker-scout.exe

Server: Containers: 18 Running: 16 Paused: 0 Stopped: 2 Images: 52 Server Version: 24.0.2 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true OSType: linux Architecture: x86_64 CPUs: 12 Total Memory: 11.7GiB Name: docker-desktop ID: e7dc7e6d-7889-45da-8028-8143d78b6bcf Docker Root Dir: /var/lib/docker Debug Mode: false HTTP Proxy: http.docker.internal:3128 HTTPS Proxy: http.docker.internal:3128 No Proxy: hubproxy.docker.internal Experimental: false Insecure Registries: hubproxy.docker.internal:5555 127.0.0.0/8 Live Restore Enabled: false

[milas]

@tr00st Thanks for the feedback! You're right that the original spec proposal and implementation are different here. We're working on reconciling things as we improve the preview functionality in Compose, so this is great.

As of now, have you been happy with the paths being relative to the Compose project directory?

[tr00st]

@milas

Not really - as per the original report, the limit to being able to only watch files inside the Compose project directory has been pretty limited when working with modular+polyrepo code. What we'd hoped for was a setup where, given a three-repo setup:

• Module 1
• App 1, uses Module 1
• App 2, also uses Module 1

Making changes in Module 1 would automatically update both App 1 and App 2, allowing the developer see the changes in either (or both) context.

Sadly, we've discontinued using Docker for most of our dev environments here since the original report, given some long-standing WSL bugs (plus poor performance on the non-WSL version under windows) making it untenable, but happy to feedback on hypotheticals for this one (I'm assuming I won't always be working on a Windows machine :) )