search

zalando / skipper

An HTTP router and reverse proxy for service composition, including use cases like Kubernetes Ingress
https://opensource.zalando.com/skipper/
Other
3.07k stars 347 forks source link

Flaky TestRedistest #2621

Open AlexanderYastrebov opened 11 months ago

AlexanderYastrebov commented 11 months ago 
2023/09/25 11:41:03 github.com/testcontainers/testcontainers-go - Connected to docker: 
  Server Version: 20.10.23
  API Version: 1.41
  Operating System: Ubuntu 20.04.6 LTS
  Total Memory: 31402 MB
  Resolved Docker Host: unix:///var/run/docker.sock
  Resolved Docker Socket Path: /var/run/docker.sock
  Test SessionID: 0bdf3f1121aaca992853190b856d6f0dea6a87d4314bfa596220d34c60b5353f
  Test ProcessID: e714cc5c-639d-423c-be15-256c609eb5f8
--- FAIL: TestRedistest (26.57s)
    redistest.go:39: Failed to start redis server: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.41/containers/a27df84a5ca5910655b167bceb730bcc3800d01be6005fe4d43a04c7111de54b/json": context deadline exceeded: creating reaper failed: failed to create container
MustafaSaber commented 11 months ago

On Mac we always face creating reaper failed: failed to create container, so we disable it via export TESTCONTAINERS_RYUK_DISABLED=true but this won't be good on CI environment I think.

AlexanderYastrebov commented 11 months ago 
--- FAIL: TestRedistest (27.04s)
    redistest.go:39: Failed to start redis server: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.41/containers/3f1705e8160379b38a506c25ff2a9ca773f1673a369040c20dfb40e5342ef067/json": context deadline exceeded: creating reaper failed: failed to create container
AlexanderYastrebov commented 11 months ago 
--- FAIL: TestRedistest (27.30s)
    redistest.go:39: Failed to start redis server: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.41/containers/02a8b90845b6d23098051cbaa2e45261b90e20901ad882254a54cac3eff807e7/json": context deadline exceeded: creating reaper failed: failed to create container
szuecs commented 11 months ago 
--- FAIL: TestRedistest (27.68s)
    redistest.go:39: Failed to start redis server: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.41/containers/9aef1c39ddc3efebaf4f754756323a5fdf702c5aa42b756a7c653a997a057fd8/json": context deadline exceeded: creating reaper failed: failed to create container
RomanZavodskikh commented 11 months ago

Moved to https://github.com/zalando/skipper/issues/2556#issuecomment-1759210053

szuecs commented 10 months ago 
--- FAIL: TestRedistest (30.99s)
    redistest.go:39: Failed to start redis server: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.41/containers/8e2c7db32998c56635b2d020fc9dbad82b1e005798894fd0fc9c746de0c01570/json": context deadline exceeded: creating reaper failed: failed to create container
FAIL
szuecs commented 10 months ago 
PASS
noleak: 2 active

goroutine 19 [IO wait]:
internal/poll.runtime_pollWait(0x7fd63a27bdb8, 0x72)
    /opt/hostedtoolcache/go/1.21.3/x64/src/runtime/netpoll.go:343 +0x85
internal/poll.(*pollDesc).wait(0xc0003e0180?, 0xc0003a3000?, 0x0)
    /opt/hostedtoolcache/go/1.21.3/x64/src/internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
    /opt/hostedtoolcache/go/1.21.3/x64/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0003e0180, {0xc0003a3000, 0x1000, 0x1000})
    /opt/hostedtoolcache/go/1.21.3/x64/src/internal/poll/fd_unix.go:164 +0x27a
net.(*netFD).Read(0xc0003e0180, {0xc0003a3000?, 0x7fcfc0?, 0xc000215e00?})
    /opt/hostedtoolcache/go/1.21.3/x64/src/net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc0003a4570, {0xc0003a3000?, 0x0?, 0xc0002fa4e0?})
    /opt/hostedtoolcache/go/1.21.3/x64/src/net/net.go:179 +0x45
net/http.(*persistConn).Read(0xc0003a7560, {0xc0003a3000?, 0xc0003ec420?, 0xc000065d38?})
    /opt/hostedtoolcache/go/1.21.3/x64/src/net/http/transport.go:1954 +0x4a
bufio.(*Reader).fill(0xc0003982a0)
    /opt/hostedtoolcache/go/1.21.3/x64/src/bufio/bufio.go:113 +0x103
bufio.(*Reader).Peek(0xc0003982a0, 0x1)
    /opt/hostedtoolcache/go/1.21.3/x64/src/bufio/bufio.go:151 +0x53
net/http.(*persistConn).readLoop(0xc0003a7560)
    /opt/hostedtoolcache/go/1.21.3/x64/src/net/http/transport.go:2118 +0x1b9
created by net/http.(*Transport).dialConn in goroutine 18
    /opt/hostedtoolcache/go/1.21.3/x64/src/net/http/transport.go:1776 +0x169f

goroutine 20 [select]:
net/http.(*persistConn).writeLoop(0xc0003a7560)
    /opt/hostedtoolcache/go/1.21.3/x64/src/net/http/transport.go:2421 +0xe5
created by net/http.(*Transport).dialConn in goroutine 18
    /opt/hostedtoolcache/go/1.21.3/x64/src/net/http/transport.go:1777 +0x16f1

Try setting GODEBUG=tracebackancestors=10 (or more), see https://pkg.go.dev/runtime#hdr-Environment_Variables
FAIL    github.com/zalando/skipper/net/redistest    6.765s
szuecs commented 10 months ago

CDP error

--- FAIL: TestRedistest (10.54s)
    redistest.go:39: Failed to start redis server: dial tcp 172.31.8.1:32768: i/o timeout: Connecting to Ryuk on 172.31.8.1:32768 failed: connecting to reaper failed: failed to create container
AlexanderYastrebov commented 6 months ago

but this won't be good on CI environment I think.

I think we actually do not needed reaper in CI, I've created #2930 to disable.