`compose run --prune`

[felipecrs]

Description

--prune would imply --rm, but would also prune every resource that got created during the run call. Including:

• Any deps (which might be running or not)
• Any volumes
• Any networks

So that it can be easily used in scenarios where I would like to spin-up a one-shot container (like an E2E tests which requires many services as dependencies), but don't want anything left in my environment after it finishes. Like during CI.

The following:

docker compose run --prune test

As a better (smarter) alternative to:

$ docker compose run --rm test

$ docker compose down

Smarter because in the second example, down would bring down everything and not only the resources that got created during the run.

Some other names for the flag are compose run --oneshot and compose run --down.

[ndeloof]

why don't you use docker compose up test if you'd like compose to manage this execution with full lifecycle support ?

[felipecrs]

I'm not sure if I understood your point...

[ndeloof]

I mean, you're asking to get the equivalent of docker compose down for your usage, so my question: why not just use it? Run your test suite using docker compose up test and the you'll benefits from compose to manage cleanup. Using run creates a one-off command which is by-design considered "orphaned".

[felipecrs]

So, you are suggesting docker compose up test; docker compose down test, right?

[ndeloof]

or docker compose up test --abort-on-container-exit so when test container complete the stack is stopped

[felipecrs]

Ok, thank you. Let me do some tests now.

[felipecrs]

Here is my example:

services:
  test-one:
    image: ubuntu:latest
    command:
      - bash
      - -c
      - |
        sleep 5s
        echo "Test succeeded"
        exit 0
    depends_on:
      - service-one
  service-one:
    image: ubuntu:latest
    command:
      - sleep
      - infinity
    depends_on:
      - prepare-one
  prepare-one:
    image: ubuntu:latest
    command:
      - bash
      - -c
      - |
        sleep 5s
        echo "Prepare succeeded"
        exit 0
  service-two:
    image: ubuntu:latest
    command:
      - sleep
      - infinity

Notice that test-one has no direct dependency on service-two. However, if service-two is already running, when issuing the down command, it will delete it too.

❯ docker compose up service-two -d
[+] Running 2/2
 ✔ Network tmpilx4slt6fi_default          Created                                                                0.1s 
 ✔ Container tmpilx4slt6fi-service-two-1  Started                                                                0.4s 
❯ docker compose up test-one --abort-on-container-exit
[+] Running 3/0
 ✔ Container tmpilx4slt6fi-prepare-one-1  Created                                                                0.0s 
 ✔ Container tmpilx4slt6fi-service-one-1  Created                                                                0.0s 
 ✔ Container tmpilx4slt6fi-test-one-1     Created                                                                0.0s 
Attaching to tmpilx4slt6fi-test-one-1
tmpilx4slt6fi-test-one-1  | Test succeeded
tmpilx4slt6fi-test-one-1 exited with code 0
Aborting on container exit...
[+] Running 1/0
 ✔ Container tmpilx4slt6fi-test-one-1  Stopped                                                                   0.0s 
❯ docker compose down
[+] Running 5/5
 ✔ Container tmpilx4slt6fi-service-two-1  Removed                                                               10.5s 
 ✔ Container tmpilx4slt6fi-test-one-1     Removed                                                                0.0s 
 ✔ Container tmpilx4slt6fi-service-one-1  Removed                                                               10.7s 
 ✔ Container tmpilx4slt6fi-prepare-one-1  Removed                                                                0.0s 
 ✔ Network tmpilx4slt6fi_default          Removed                                                                0.8s

My intention was that, with docker compose up test-one --abort-on-container-exit --prune, it could remember which services it had to start for running test-one, and once test-one exits, it would tear down everything it had to start (which in my example above includes test-one, service-one and prepare-one, but not service-two).

[ndeloof]

it would tear down everything it had to start

this is not compose philosophy. You describe a desired state as yaml, compose creates resources accordingly and remove (all of) them at once. Can you maybe explain your requirements? What's the benefit for you to keep this service running after command completion?

[felipecrs]

My use case is the following:

• I have several services, and several tests either
• They are all mixed in the same docker-compose.yaml
• For example, test-one, test-two, test-three, test-four
• Each test has some specific dependencies
• But all tests depends on some certain dependencies (common dependencies across all tests)
• I want to use docker compose for running these tests in CI

For example:

1. Setup common dependencies: docker compose up common-dependencies
2. Test one: docker compose run --rm test-one, which starts its specific dependencies, then tears it down to free up resources
3. Test two: docker compose run --rm test-two, which starts its specific dependencies, then tears it down to free up resources
4. Several other similar tests

My current workaround is to hardcode all the specific dependencies during each test cleanup:

2. docker compose rm --force --volumes prepare-one service-one
3. docker compose rm --force --volumes prepare-two service-two

[ndeloof]

That's an interesting use case. There's no simple solution for now to cover this, but I had in mind we could introduce support for "composition of compose projets" for large applications, where some subcomponents could be declared in a compose file, but would require another set of components to be running, as a separate compose project:

services:
  my-service:
    image: ....

require:
   core-app: ../app/compose.yaml

which such a declaration, docker compose up would trigger a child docker compose up command on required compose app, and would be able to refer to it's resources. But other lifecycle commands would only consider the local application.

Just an idea to be debated and experimented :)

[felipecrs]

That's interesting... 🙂

[ndeloof]

see https://github.com/compose-spec/compose-spec/pull/363