This is a good feature that lends itself to a GitOps approach to remote Docker Compose deployments. However I think this should be a toggle, especially with secrets. Yes

[Bman7714]

This is a good feature that lends itself to a GitOps approach to remote Docker Compose deployments. However I think this should be a toggle, especially with secrets.

For example, in my workflow I have various docker-compose.ymls each in different directories, with each directory corresponding to a remote server. As a very basic example:

infrastructure-as-code
├── dev
│   └── instances
│       ├── dev-server-1
│       │   └── docker-compose.yml
│       └── dev-server-2
│           └── docker-compose.yml
└── prod
    └── instances
        ├── prod-server-1
        │   └── docker-compose.yml
        └── prod-server-2
            └── docker-compose.yml

This is all Git tracked as a single source of truth to track server state and changes to deployments over time. By being able to use configs from my localhost, we can also track Docker config changes in Git. However, it doesn't make sense to track secrets in the Git repository, as they are meant to be secret - it would be more ideal for my workflow to bind/copy secrets from DOCKER_HOST.

I think an attribute such as from_docker_host would be useful, so you can explicitly choose where the config/secret files will be found when using Docker contexts, and it is flexible around different workflows people may use:

configs:
  my_config:
    file: ./service.conf
    from_docker_host: false # Will search my localhost for service.conf

secrets:
  my_secret:
    file: /etc/my-docker-secerts/private.pem
    from_docker_host: true # Will search DOCKER_HOST for /etc/my-docker-secerts/private.pem

Originally posted by @LawrenceWarren in https://github.com/docker/compose/issues/11867#issuecomment-2147007301

[ndeloof]

This demonstrates lack of native support for secrets in docker engine (when swarm mode is disabled) which require users to be creative :) Reminds me an earlier discussion we had regarding secrets extensibility (cc @hyu). The topic was basically "how to let compose retrieve secrets from an external service like Vault". What you implemented here is basically a hand-made secret provider. Would be nice you can fully delegate secret management to some custom component so you get my_secret eventually bound to a path on docker host.

Regarding this proposal, using such a from_docker_host would require we disable relative path resolution on file (which takes place client side and would not make sense). Such cross-attribute logic would be a pain to implement. IMHO if we want to explicitly cover this usage, we would better add support for a dedicated host_path (actual name to be debated) attribute, that would be exclusive vs file and environment, and would be documented to disable support for uid, gid andmode`. Such an attribute would not be considered relative to project directory, and passed as-is to docker engine.