search

testcontainers / testcontainers-java

Testcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.
https://testcontainers.org
MIT License
7.91k stars 1.62k forks source link

[Bug]: Can't connect to Ryuk when running in container on Jenkins #8577

Open abderrezakBoukazouha opened 3 months ago

abderrezakBoukazouha commented 3 months ago

Module

Core

Testcontainers version

1.19.7

Using the latest Testcontainers version?

Yes

Host OS

CentOS Linux 7 (Core)

Host Arch

Linux 3.10.0-327.36.1.el7.x86_64

Docker version

Client: Docker Engine - Community
 Version:           24.0.5
 API version:       1.43
 Go version:        go1.20.6
 Git commit:        ced0996
 Built:             Fri Jul 21 20:39:02 2023
 OS/Arch:           linux/amd64
 Context:           default

What happened?

I'm Trying to run some tests with TestContainers inside a Jenkins pipeline, with those two command : docker build -f Dockerfile-package-application -t build-container . --progress=plain --network=host

docker run -e TESTCONTAINERS_HOST_OVERRIDE=localhost --network=host -v /etc/passwd:/etc/passwd:ro --rm -v ${playlist_services_directory}:${playlist_services_directory} -w ${playlist_services_directory} -v /var/run/docker.sock:/var/run/docker.sock -t build-container

and the dockerfile has a basic CMD ["mvn", "clean", "install" ]

--network=host is for allowing the containers to upload the maven dependancies via a Mirror

The launch of the test in Intellij or in a docker (In Macbook pro M1) is very smooth and get the result I want, but not on Jenkins, where I receive those Logs, where ryuk tries a couple of times before it crashes. Note that I used different strategies like -e TESTCONTAINERS_HOST_OVERRIDE=host or -e TESTCONTAINERS_HOST_OVERRIDE=172.17.0.1 but the issue remains only on Jenkins. it a question of networking a believe and it's a real Struggle.

Relevant log output

12:59:26.992 [main] INFO org.testcontainers.dockerclient.DockerClientProviderStrategy -- Found Docker environment with local Unix socket (unix:///var/run/docker.sock)
12:59:27.009 [main] INFO org.testcontainers.DockerClientFactory -- Docker host IP address is localhost
12:59:27.029 [main] INFO org.testcontainers.DockerClientFactory -- Connected to docker: 
  Server Version: 24.0.5
  API Version: 1.43
  Operating System: CentOS Linux 7 (Core)
  Total Memory: 29982 MB
12:59:27.061 [main] INFO tc.testcontainers/ryuk:0.6.0 -- Pulling docker image: testcontainers/ryuk:0.6.0. Please be patient; this may take some time but only needs to be done once.
12:59:27.066 [main] INFO org.testcontainers.utility.RegistryAuthLocator -- Failure when attempting to lookup auth config. Please ignore if you don't have images in an authenticated registry. Details: (dockerImageName: testcontainers/ryuk:latest, configFile: /root/.docker/config.json, configEnv: DOCKER_AUTH_CONFIG). Falling back to docker-java default behaviour. Exception message: Status 404: No config supplied. Checked in order: /root/.docker/config.json (file not found), DOCKER_AUTH_CONFIG (not set)
12:59:28.038 [docker-java-stream--623331358] INFO tc.testcontainers/ryuk:0.6.0 -- Starting to pull image
12:59:28.062 [docker-java-stream--623331358] INFO tc.testcontainers/ryuk:0.6.0 -- Pulling image layers:  0 pending,  0 downloaded,  0 extracted, (0 bytes/0 bytes)
12:59:28.647 [docker-java-stream--623331358] INFO tc.testcontainers/ryuk:0.6.0 -- Pulling image layers:  2 pending,  1 downloaded,  0 extracted, (311 KB/? MB)
12:59:28.717 [docker-java-stream--623331358] INFO tc.testcontainers/ryuk:0.6.0 -- Pulling image layers:  1 pending,  2 downloaded,  0 extracted, (3 MB/? MB)
12:59:28.723 [docker-java-stream--623331358] INFO tc.testcontainers/ryuk:0.6.0 -- Pulling image layers:  0 pending,  3 downloaded,  0 extracted, (3 MB/6 MB)
12:59:29.194 [docker-java-stream--623331358] INFO tc.testcontainers/ryuk:0.6.0 -- Pulling image layers:  0 pending,  3 downloaded,  1 extracted, (3 MB/6 MB)
12:59:29.439 [docker-java-stream--623331358] INFO tc.testcontainers/ryuk:0.6.0 -- Pulling image layers:  0 pending,  3 downloaded,  2 extracted, (3 MB/6 MB)
12:59:29.687 [docker-java-stream--623331358] INFO tc.testcontainers/ryuk:0.6.0 -- Pulling image layers:  0 pending,  3 downloaded,  3 extracted, (6 MB/6 MB)
12:59:29.708 [docker-java-stream--623331358] INFO tc.testcontainers/ryuk:0.6.0 -- Pull complete. 3 layers, pulled in 1s (downloaded 6 MB at 6 MB/s)
12:59:29.708 [main] INFO tc.testcontainers/ryuk:0.6.0 -- Image testcontainers/ryuk:0.6.0 pull took PT2.646862225S
12:59:29.730 [main] INFO tc.testcontainers/ryuk:0.6.0 -- Creating container for image: testcontainers/ryuk:0.6.0
12:59:30.028 [main] INFO tc.testcontainers/ryuk:0.6.0 -- Container testcontainers/ryuk:0.6.0 is starting: bfe8f38ba2a40ee6fcb1766a5625e2b73882a6b1fdeb8c07db4100cea9d5ff1d
12:59:30.362 [main] INFO tc.testcontainers/ryuk:0.6.0 -- Container testcontainers/ryuk:0.6.0 started in PT0.631759785S
12:59:33.372 [testcontainers-ryuk] WARN org.testcontainers.utility.RyukResourceReaper -- Can not connect to Ryuk at localhost:32797
java.net.SocketException: Connection reset
    at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:318)
    at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
    at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
    at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1099)
    at java.base/sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:350)
    at java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:393)
    at java.base/sun.nio.cs.StreamDecoder.lockedRead(StreamDecoder.java:217)
    at java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:171)
    at java.base/java.io.InputStreamReader.read(InputStreamReader.java:188)
    at java.base/java.io.BufferedReader.fill(BufferedReader.java:160)
    at java.base/java.io.BufferedReader.implReadLine(BufferedReader.java:370)
    at java.base/java.io.BufferedReader.readLine(BufferedReader.java:347)
    at java.base/java.io.BufferedReader.readLine(BufferedReader.java:436)
    at org.testcontainers.utility.ResourceReaper$FilterRegistry.waitForAcknowledgment(ResourceReaper.java:397)
    at org.testcontainers.utility.ResourceReaper$FilterRegistry.register(ResourceReaper.java:393)
    at org.testcontainers.utility.RyukResourceReaper.lambda$null$1(RyukResourceReaper.java:122)
    at org.rnorth.ducttape.ratelimits.RateLimiter.doWhenReady(RateLimiter.java:27)
    at org.testcontainers.utility.RyukResourceReaper.lambda$maybeStart$2(RyukResourceReaper.java:101)
    at java.base/java.lang.Thread.run(Thread.java:1583)

Additional Information

Note : I'm not Able to reproduce this behaviour locally, my build and tests with the same command works perfectly No response

xstefank commented 3 weeks ago

We have the same issue.

EDIT: We had to workaround this with TESTCONTAINERS_RYUK_DISABLED=true.

abderrezakBoukazouha commented 3 weeks ago

I believe it's related to Jenkins not allowing containers to connect between them with Ip adresses. Most likely a networking issue