Support init scripts for `MongoDBContainer` without manually customizing the `WaitStrategy`

[blaluc]

A container initialized as follows, fails to start:

MongoDBContainer c = new MongoDBContainer(MONGO_IMAGE)
        .withEnv("MONGO_INITDB_ROOT_USERNAME", "root")
        .withEnv("MONGO_INITDB_ROOT_PASSWORD", "password" )
        .withFileSystemBind("init/mongo-init.js",
            "/docker-entrypoint-initdb.d/mongo-init.js", BindMode.READ_ONLY)
        ;
        c.start()

Not sure but probably because the docker-entrypoint.sh, in case of presence of init-db scripts, does a shutdown and a restart of the process after executing them:

if [ -n "$shouldPerformInitdb" ]; then
...
        echo
        for f in /docker-entrypoint-initdb.d/*; do
            case "$f" in
                *.sh) echo "$0: running $f"; . "$f" ;;
                *.js) echo "$0: running $f"; "${mongo[@]}" "$MONGO_INITDB_DATABASE" "$f"; echo ;;
                *)    echo "$0: ignoring $f" ;;
            esac
            echo
        done

        "${mongodHackedArgs[@]}" --shutdown
        rm -f "$pidfile"
...

[vcvitaly]

Is it something that should be raised with https://github.com/docker-library/mongo ?

[kiview]

This is indeed somewhat related to #3077, but more in the way, that MongoDBContainer uses a LogMessageWaitStrategy that has certain expectations about the logs.

You can solve this by configuring your own appropriate LogMessageWaitStrategy.

[blekione]

@blaluc Did you manage to start the container with an init script? I'm facing the same issue, tried to use wait for log message configuration but unsuccessful.

[kiview]

@blekione Which LogMessageWaitStrategy configuration did you use?

[blekione]

 Wait.forLogMessage("your_log_message", 1)

So I added to my init script print statement with message which I'm waiting for to be printed. Most likely I'm doing it wrong, as I can see this message in logs when checking logs directly in container with docker logs but not in my test logs.

I'm really testing the grounds here. Never used Testcontainers with MongoDB

[kiview]

Can you please share a reproducer? https://github.com/testcontainers/testcontainers-java-repro

[blekione]

https://github.com/blekione/testcontainers_reprod

@kiview or should I create pull request?

[kiview]

This is sufficient, thanks.

I am not sure what you are trying to ultimately achieve, but changing the container code like this, will be enough to wait for the log statement:

MongoDBContainer mongoDb  = new MongoDBContainer("mongo:5.0")
        .withCopyFileToContainer(MountableFile.forHostPath("init/mongo-init.js"), 
            "/docker-entrypoint-initdb.d/mongo-init.js")
        .waitingFor(Wait.forLogMessage("Start_script_now\\s", 1))
        .withStartupTimeout(Duration.ofSeconds(10));

Note the \s flag for the matching of whitespace character (newline) at the end of the line.

However, this will fail now with the initialization of the replica set. Probably because you overwrote the actual initialization scrit like this? (not a MongoDB expert, sorry)

[blekione]

@kiview I want to add some records to the database at the initialisation. I tend to write first test as if my db service can read from the database before I can write (chicken and egg problem really). Using init script is the only way I know to do so. Not expert from MongoDB either, and I suspect much less knowledgable than you with Docker and Testcontainers.

[kiview]

Oh sorry, I understand the context now better (this is is 1,5 years old after all).

This here is the example using the appropriate log message if an init script is used:

MongoDBContainer mongoDb  = new MongoDBContainer("mongo:5.0")
    .withCopyFileToContainer(MountableFile.forHostPath("init/mongo-init.js"),
            "/docker-entrypoint-initdb.d/mongo-init.js")
    .waitingFor(Wait.forLogMessage("(?i).*waiting for connections.*", 2))
    .withStartupTimeout(Duration.ofSeconds(10));

Placing the init script into /docker-entrypoint-initdb.d will trigger a restart of MongoDB. So by changing the LogMessageWaitStrategy to expect the default log message 2 times will make this setup work. You can now fill mongo-init.js with your specific code to fill the database.

I also decided to re-open the issue and rephrase it, since I feel this is a lacking feature from our MongoDBContainer implementation.

[selberget]

Is anyone currently working on this? I can give this a try otherwise.

As mentioned above, I noticed that you'd to include rs.initiate() as part of the initialization scripts (and wait until the instance is master) to be able to execute write commands in the scripts, otherwise it fails with:

> db.createCollection("images")
{
    "operationTime" : Timestamp(0, 0),
    "ok" : 0,
    "errmsg" : "not master",
    "code" : 10107,
    "codeName" : "NotMaster",
    "$clusterTime" : {
        "clusterTime" : Timestamp(0, 0),
        "signature" : {
            "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
            "keyId" : NumberLong(0)
        }
    }
}

Could a possible solution to this be to add an additional init script that includes the rs.initiate() and wait for master command (from buildMongoWaitCommand()), and make sure that it gets executed prior to the user provided script (executed in alphabetic order)?

withCopyToContainer(Transferable.of(buildMongoInitReplicaSetScript()), "/docker-entrypoint-initdb.d/init-rs.js");

withCopyFileToContainer(initScript, "/docker-entrypoint-initdb.d/mongo-init.js")

And then skip to execute rs.initiate() command when container has started but still issue the wait for master command.

One problem with this is how to present a proper error message if the rs.initiate() command fails in the script.

Or maybe the responsibility of initiating the replica set should lie with the user who adds the script?

[kiview]

@selberget No one is currently working on this. TBH, I did not fully get your proposal. Why deal with a custom init script and not just adapt the WaitStrategy to robustly cover this scenario?

[selberget]

Sorry @kiview, I will try to explain myself more clearly.

If I don't include rs.initiate() and wait for master command in the init script, this fails after the first .*waiting for connections.* log message.

// rs.initiate()
// while (db.runCommand( { isMaster: 1 } ).ismaster==false) { sleep(100); }
db.images.insertMany( [
  { name: "mongo", tag: "4.4" },
  { name: "mongo", tag: "4.0.10" }
] );

I'm not really sure on how I can use WaitStrategy to handle this scenario, i.e. making ìnitReplicaSet() method be executed prior to the init script and before the container restart.

But this is only relevant if it should be something that should automatically be handled by the module, maybe a user that wants to provide a init script to a replica set should be aware of these steps.

I'm not an mongodb expert so maybe there are some other ways to populate the data from the script, without having to issue the rs.initiate() command.

[FlorinAlexandru]

I've encountered a similar need. I even asked a SO question: https://stackoverflow.com/questions/75927412/unable-to-add-init-script-to-mongo-in-testcontainers

When adding anything in the init script, the test is failing with configuration exception. I think that the script is run before the replica is register to replica set.

[Abhi-y2003]

Hey is this issue fixed, can you assign it to me .