Open aakso opened 1 month ago
Same issue, solved with:
// HOTFIX: testcontainers-go v0.32.0
// https://github.com/testcontainers/testcontainers-go/issues/2670
type hotfix2670 struct{}
func (h hotfix2670) Customize(req *testcontainers.GenericContainerRequest) error {
originalHook := req.LifecycleHooks[0].PostStarts[0]
req.LifecycleHooks[0].PostStarts[0] = func(ctx context.Context, container testcontainers.Container) error {
var err error
for retry := 0; retry < 10; retry++ {
err = originalHook(ctx, container)
if err == nil {
break
}
time.Sleep(time.Second)
}
return err
}
return nil
}
func setupKafka(t *testing.T, ctx context.Context) *kafka.KafkaContainer {
t.Helper()
testcontainers.Logger = log.New(io.Discard, "", 0)
kafkaContainer, err := kafka.Run(ctx, "confluentinc/confluent-local:7.5.0",
kafka.WithClusterID("test-cluster"),
hotfix2670{},
)
require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, kafkaContainer.Terminate(ctx)) })
return kafkaContainer
}
@ar-sematell yeah, my solution was very similar. Unfortunately, it only tackles the LifecycleHooks part.
The second problem with GenericContainer
can probably be mitigated only with retrying the entire operation.
Can you check if https://github.com/testcontainers/testcontainers-go/pull/2696 fixes this?
@stevenh I tested with pseudo module version of v0.32.1-0.20240814110719-4545c292e7b8
and it seems the issue is resolved.
I tried with few batches of 10 and with v0.32.0, about 50% of container starts failed. With the new patch, I don't see any failures.
@aakso we released v0.33.0
Could you please check with that version?
If fixed, I think we can close this one, thanks!
Unforunately I get: "create container: port not found: creating reaper failed" errors with both v0.32.1-0.20240814110719-4545c292e7b8 and v0.33.0
Looks like this will need to reaper rework, do you have a simple test case?
Unfortunately the test case is relatively complicated, it will take a bit of effort to boil it down to a simpler examplar
Testcontainers version
0.32.0
Using the latest Testcontainers version?
Yes
Host OS
MacOS
Host arch
ARM
Go version
1.21.12
Docker version
Docker info
What happened?
related to #2543
Unfortunately, the "port not found" error still seems to be present with v0.32.0.
I see it mainly with the Kafka module but also occasionally with
testcontainers.GenericContainer
too.The code snippet I'm using the start the Kafka test container:
So pretty standard.
Relevant log output
Additional information
I'm able to mitigate the error for Kafka module by wrapping the
PostStarts
lifecycle hooks and retrying on error. In my case it's the first hook that always fails. It seems to fail toc.MappedPort
call. So far, a single retry has been enough to mitigate the problem.