Closed FRosner closed 1 week ago
Thanks for reporting @FRosner, we'll check in with the colleagues maintaining Docker, whether something might have changed in this regard.
It is also surprising, that our GHA workflows don't fail in this case (I would assume they also updated, but @eddumelendez can confirm).
@kiview nice to see you again :P
It's not failing for single tests. It seems to be a race condition as we create hundreds of networks concurrently in our tests across multiple JVMs.
As a data point, I see the same issue using the latest Testcontainers version.
Core
1.19.8
Yes
Ubuntu 24.04
x86
Client: Docker Engine - Community
Version: 27.0.2
API version: 1.46
Go version: go1.21.11
Git commit: 912c1dd
Built: Wed Jun 26 18:47:25 2024
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 27.0.2
API version: 1.46 (minimum version 1.24)
Go version: go1.21.11
Git commit: e953d76
Built: Wed Jun 26 18:47:25 2024
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.7.18
GitCommit: ae71819c4f5e67bb4d5ae76a6b735f29cc25774e
runc:
Version: 1.7.18
GitCommit: v1.1.13-0-g58aa920
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Hi all - I think it's this Docker Engine regression ... https://github.com/moby/moby/issues/48069#issuecomment-2195563789
I was wondering if this is maybe a regression in docker itself, and it's possible, but I wasn't able to reproduce locally with `for n in $(seq 0 256); do docker network create "n$n"& done.
It's not failing for single tests. It seems to be a race condition as we create hundreds of networks concurrently in our tests across multiple JVMs.
I don't think it'll be reproducible with just-allocations, it'll happen with a mix of network creation and deletion. Not a timing race, it's caused by an ordered list getting out of order.
I'm working on a fix - but if this description doesn't fit what you're seeing, please let me know.
Thanks! So if there's nothing testcontainers can do, let's close this issue?
Closing this issue. The fix should land in the next docker versions and let's consider that we should also wait once GH runners has been updated.
Module
Core
Testcontainers version
1.19.1
Using the latest Testcontainers version?
No
Host OS
Linux
Host Arch
x86, ARM
Docker version
What happened?
All of a sudden many of our GitHub action workflows started failing, because our tests couldn't start the containers. The error we see in the logs is:
Relevant log output
No response
Additional Information
This has started only after Ubuntu published the new docker version 27 to its repository and our runners upgraded automatically, because we didn't pin the version. I see in the release notes that there have been some changes to networking (and some networking APIs), so I'm wondering if this introduced some race condition.
In our tests, we create many networks concurrently using
Network.newNetwork()
, and that worked fine so far, as the docker daemon should be assigning subnets in an incrementing fashion.I was wondering if this is maybe a regression in docker itself, and it's possible, but I wasn't able to reproduce locally with `for n in $(seq 0 256); do docker network create "n$n"& done.