Support "docker exec [command]" as readiness checks for containers in Docker Compose

[igormukhin]

Spring Boot Version: 3.2.x Component: Spring Boot Docker Compose Issue type: Enhancement

Problem case:

If you have a MongoDB container that needs to be single-node replica set (e.g. to support transaction), you need to execute some commands on the started Mongo container to configure the replica set. Then, you need to wait for some other specific command to ensure that the replica set was fully initialized.

This how Testcontainers does it: Using docker exec they run the command sh -c mongosh mongo --eval [JAVASCRIPT] || mongo --eval [JAVASCRIPT] where [JAVASCRIPT] is:

• first, rs.initiate() to start the replica set
• second, var attempt = 0; while(db.runCommand({ isMaster: 1 }).ismaster == false) { if (attempt > 60) { quit(1); } sleep(100); attempt++; } to wait for the replica set to initialize

Currently, this behaviour is very hard (impossible?) to implement with Spring Boot Docker Compose.

General Problem:

Some docker containers may require custom wait on start strategy. Currently, Spring Boot only waits for a TCP port to become responsive.

Proposal

Implement additional wait on strategy, where users can provide a shell command in an environment variable:

env:
  SPRING_BOOT_DOCKER_COMPOSE_READINESS_CHECK_COMMAND: sh -c mongosh mongo --eval "if(db.adminCommand({replSetGetStatus: 1})['myState'] != 1) quit(666)"

If a container has this variable set, Spring Boot should execute this command X number of times until its exit code becomes 0.

[wilkinsona]

Thanks for the suggestion.

Generally speaking, we consider Testcontainer and Docker Compose to exist at the same level in the stack, both offering a layer of additional functionality on top of Docker. A such, I think it would be more appropriate for Docker Compose to offer the functionality that you have described, rather than Spring Boot. This would bring the benefits to everyone using Docker Compose and not just those using Spring Boot and Docker Compose. Alternatively, the support could even move down in the stack and be offered by Mongo's Docker container. This would then benefit everyone using Mongo and Docker, broadening its audience even further.

Arguably, Docker Compose already has the support that you're looking for in the form of its healthcheck that can be used to create a replica set. Have you explored this approach at all? I would be interested to know if it works with Spring Boot's Docker Compose integration. If it does not, I think that may be something that we'd want to look at.

[igormukhin]

@wilkinsona, thanks you!

I already looked into the "healthcheck"-trick which helps to configure the replica set without using custom Dockerfile, but this trick does not help for clients to wait until the container is fully initialized.

In our project we are using Testcontainers to run docker containers both for running the application locally and for running integration tests. We have an additional LocalStackApplication (imports LocalStackConfiguration) in the test sources which a developer needs to start before running the MainApplication itself. LocalStackConfiguration starts Testcontainers, WireMock servers and other stuff. And for integration tests, we just put @Import(LocalStackConfiguration.class) on the test class to initialize the local stack. This is working fine.

Last week I needed to create a small demo project and Spring Boot Docker Compose was very helpful for running a Postgres database quickly. So I decided to try it out on the main project. And the limitation of the waiting strategy is one of the issues I ran into.

The waiting strategy is not an issue for just running the application locally, but it is an issue for the automated tests - they really need all the infrastructure to be ready before starting with the tests.

I think I also have an idea for a workaround:

@Configuration
@Profile("localstack")
public class LocalStackConfiguration {

    @EventListener(DockerComposeServicesReadyEvent.class)
    public void onDockerComposeReady(DockerComposeServicesReadyEvent event) {
        findMongoContainer(event.getRunningServices()).ifPresent(this::configureAndWaitOnReplicaSet);
    }

    private void configureAndWaitOnReplicaSet(RunningService mongo) {
        // TODO: run `docker exec` like Testcontainers does
    }

[mhalbritter]

but this trick does not help for clients to wait until the container is fully initialized.

I don't quite understand. We call docker compose up --detach --wait. According to docker compose help:

--wait                      Wait for services to be running|healthy. Implies detached mode.

So the up command should only return control back to Spring Boot after the container is healthy, if a healthcheck has been configured on the service. And only after docker compose up has returned, we try our TCP healthcheck.

So if you put sh -c mongosh mongo --eval [JAVASCRIPT] || mongo --eval [JAVASCRIPT] as healthcheck on your service, it should work. Is this the setup you've tried?

[igormukhin]

@mhalbritter, thanks, I didn't know about the "--wait" flag. I just tried out the following configuration:

  mongodb:
    image: "mongo:7.0.6"
    command: ["--replSet", "rs0"]
    ports:
      - "27017:27017"
    healthcheck:
      test: echo "try { rs.status() } catch (err) { rs.initiate() }" | mongosh mongo --quiet
      interval: 5s
      timeout: 30s
      retries: 30

It works!

The issue can be probably closed.

[mhalbritter]

Great to hear!