testcontainers / testcontainers-rs

A library for integration-testing against docker containers from within Rust.
https://rust.testcontainers.org
Apache License 2.0
745 stars 142 forks source link

test: deflake the test `start_containers_in_parallel` #748

Closed hovinen closed 2 weeks ago

hovinen commented 2 weeks ago

The test often fails, especially in GitHub actions. The test attempts to start four containers on the same image and sets a deadline to prove that they start in parallel rather than in serial. The failure messages indicate tha this deadline is exceeded.

This appears to happen when the image of the container being started has to be pulled. Sometimes the pull operation takes so much time that the deadline is exceeded just due to that.

This adds some lines to the test which start and then stop a container on the same image before attempting to start the containers in parallel. Doing so ensures that the image has already been pulled when the real test starts running. Then the containers should start in close to the intended two seconds, so the test should pass more reliably.

Manual experiments appear to support this thesis: Before this change, running

$ docker image rm hello-world
$ cargo test --test async_runner --no-default-features --features blocking start_containers_in_parallel

would consistently fail due to an exceeded deadline. With this change, the test consistently passes.

Note: This and other tests still depend on access to the Docker Hub and fail when they cannot pull the images they need. This can also lead to flakiness.

netlify[bot] commented 2 weeks ago

Deploy Preview for testcontainers-rust ready!

Name Link
Latest commit 6d488f4baa9d6498be85df2be9282f3c5dae5929
Latest deploy log https://app.netlify.com/sites/testcontainers-rust/deploys/67160e9ad0a6920008cd4f44
Deploy Preview https://deploy-preview-748--testcontainers-rust.netlify.app
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

DDtKey commented 2 weeks ago

Thank you for the contribution! 👍 In this test, it makes sense to pull the image in advance, because we start them in parallel.

I've left just one comment to pull the image in more appropriate way.