Feature request: docker stack rm --compose-file option

[domg123]

Description docker stack deploy is additive, in that ... if I have uniquely named services within 2 compose files, after this command ...

docker stack deploy --compose-file service_1.yml my-stack

... I will have one service, in one stack...

NAME       SERVICES
my-stack  1

and if I then run a different compose file, with a different service name...

docker stack deploy --compose-file service_2.yml my-stack

I will still have the one stack, but 2 services.

NAME       SERVICES
my-stack  2

Now, this is a really useful pattern, as you get some of the benefits of the stack (the namespacing, the idempotent deploy cmd), but you can define each service separately, and update them separately etc.

But the reverse doesn't work, i.e. I can't do ...

docker stack remove --compose-file service_1.yml my-stack

.. and just remove the services in the given compose file back off the stack (leaving those added with the _2.yml file). The above just yields an error, as there is no "--compose-file" flag for remove.

I can individually remove the services in the compose file, by name, and get the same result, but then I can't take advantage of all the service/nw names being defined already in the compose file.

The above was based on how I observed it to work in 17.04.0-ce-rc1

[thaJeztah]

We merged a pull request that adds a --prune option to docker stack deploy. When using that option, services that are no longer defined in the docker-compose file will be removed when deploying.

It didn't make it for 17.04, but will be in 17.05; details are in the pull request; https://github.com/docker/docker/pull/31302

Does that solve your use case?

[domg123]

I don't think it quite does.

So, my use-case is that I have some number of services (let's say 3), that I wish to use in one stack essentially for multi-tenancy. I am going to create multiple stacks (i.e. one per customer) with the same 3 services each time.

That said, I also like them being kept in seperate compose files, it lets me update a service independent of the rest of the stack, and is useful when the total app is really a set of modular services, and each individual service has a separate pipeline/lifecycle. and I don't want to be forced to define them all in the one place.

The prune example is essentially the reverse, remove everything not defined. Where I want "remove everything defined" in the dc file. This is the mirror of how deploy works, which creates everything that is defined, and will leave existing services alone..

[thaJeztah]

I don't think that's really the design of the feature; the compose file should be a full definition of the stack; the usage of deploying stacks after each other to complement the previous one may be relying on an omission that services that are no longer defined aren't automatically removed.

/cc @dnephin who may be able to confirm

[domg123]

I thought perhaps that was the case.

Can I suggest it is a useful feature though. Perhaps if 'docker stack deploy' is altered to mirror 'docker stack remove', and is no longer additive, maybe an additional flag (--ignore-undefined, or something better) can be used to specify that services not in the referenced compose file, in a docker stack deploy|remove command, will be left alone.

[dnephin]

@thaJeztah is correct. A stack should be considered an atomic unit.

It sounds what you're looking for is some way to configure individual services from a config file, along with a "namespace" to group things together. That would be a great feature, but it's a different problem from the one solved by stacks.

[domg123]

I think that is fair, an alternative approach could be adding a --compose-file and --namespace flag to the existing docker service deploy/update/rm

I'd add that the nice idempotency of the 'stack deploy' is missing from these commands as they stand, I'm not sure if that can be cleanly added to the existing docker service deploy, so that it will do an update, if the service exists (and not just fail).

[domg123]

..or alternatively, there could be more control of the name spacing of a stack (afaik you can't control this, but I recall compose used to allow you to change it with an environment variable). Then I can have a stack for each service, they are independent, but I can control the naming prefix at deployment, so that they share the name space as other stacks

[MetalArend]

the usage of deploying stacks after each other to complement the previous one may be relying on an omission that services that are no longer defined aren't automatically removed

Having to add the --prune parameter to the deploy command might make it clear that at this moment a stack is not managed as described there. There is some unclear behaviour between the deploy and the other commands. If your stack is an atomic unit, what exactly is that "atomic unit", and where is that atomic unit being defined? In the docker-compose file? Or in the actual stack? By the labels on the services? Or simply by all services running at that time? It seems this behaviour could be more clear, as the deploy and the rm commands are behaving differently, and thus either way incoherent.

Using the docker-compose file as your source of truth seems to me to be the most easy way to go here. And it makes it very clear on what to handle with any command being run. The docker-compose file tells you exactly which services/scope to handle with the command that is referencing the file.