search

TykTechnologies / tyk

Tyk Open Source API Gateway written in Go, supporting REST, GraphQL, TCP and gRPC protocols
Other
9.73k stars 1.09k forks source link

[TT-13142] SSE Streaming is broken (follow up to TT-12318) #6554

Open alekc opened 1 month ago

alekc commented 1 month ago

Original bug (https://github.com/TykTechnologies/tyk/issues/6322) is marked as solved, however it's not the case.

If the response is coming from SSE streaming, AND record detailed statistics is enabled, then it would crash out with panic. 

2024/09/19 17:32:03 http: panic serving [::1]:53263: runtime error: invalid memory address or nil pointer dereference
goroutine 265 [running]:
net/http.(*conn).serve.func1()
        /Users/alexander.chernov/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.6.darwin-arm64/src/net/http/server.go:1903 +0x118
panic({0x1060041a0?, 0x1071d5260?})
        /Users/alexander.chernov/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.6.darwin-arm64/src/runtime/panic.go:770 +0xf0
io.ReadAll({0x0, 0x0})
        /Users/alexander.chernov/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.6.darwin-arm64/src/io/io.go:712 +0xb8
github.com/TykTechnologies/tyk/gateway.(*SuccessHandler).RecordHit(0x14000eb9c20, 0x14000614d80, {0xa850, 0x72}, 0xc8, 0x14000131b00)
        /Users/alexander.chernov/dev/genai/go/tyk/gateway/handler_success.go:223 +0x5f4
github.com/TykTechnologies/tyk/gateway.(*SuccessHandler).ServeHTTP(0x14000eb9c20, {0x10630e7b0, 0x140000f2000}, 0x14000614d80)
        /Users/alexander.chernov/dev/genai/go/tyk/gateway/handler_success.go:380 +0x408
github.com/TykTechnologies/tyk/gateway.(*DummyProxyHandler).ServeHTTP(0x14000eb9c20, {0x10630e7b0, 0x140000f2000}, 0x14000614d80)
        /Users/alexander.chernov/dev/genai/go/tyk/gateway/api_loader.go:652 +0xf08
github.com/TykTechnologies/tyk/gateway.(*Gateway).createMiddleware.func1.1({0x10630e7b0, 0x140000f2000}, 0x14000614d80)
        /Users/alexander.chernov/dev/genai/go/tyk/gateway/middleware.go:201 +0xef0
net/http.HandlerFunc.ServeHTTP(0x14000ed8000, {0x10630e7b0, 0x140000f2000}, 0x14000614d80)
        /Users/alexander.chernov/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.6.darwin-arm64/src/net/http/server.go:2171 +0x40
github.com/TykTechnologies/tyk/gateway.(*Gateway).createMiddleware.func1.1({0x10630e7b0, 0x140000f2000}, 0x14000614d80)
        /Users/alexander.chernov/dev/genai/go/tyk/gateway/middleware.go:201 +0xef0
net/http.HandlerFunc.ServeHTTP(0x14000ed8050, {0x10630e7b0, 0x140000f2000}, 0x14000614d80)
        /Users/alexander.chernov/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.6.darwin-arm64/src/net/http/server.go:2171 +0x40
github.com/TykTechnologies/tyk/trace.Handle.func1({0x10630e7b0, 0x140000f2000}, 0x14000614a20)
        /Users/alexander.chernov/dev/genai/go/tyk/trace/handler.go:11 +0x148
net/http.HandlerFunc.ServeHTTP(0x14000ed24e0, {0x10630e7b0, 0x140000f2000}, 0x14000614a20)
        /Users/alexander.chernov/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.6.darwin-arm64/src/net/http/server.go:2171 +0x40
github.com/gorilla/mux.(*Router).ServeHTTP(0x140003fd200, {0x10630e7b0, 0x140000f2000}, 0x14000614a20)
        /Users/alexander.chernov/go/pkg/mod/github.com/gorilla/mux@v1.8.1/mux.go:212 +0x254
github.com/TykTechnologies/tyk/gateway.(*handleWrapper).ServeHTTP(0x140002dcee0, {0x10630e7b0, 0x140000f2000}, 0x140006147e0)
        /Users/alexander.chernov/dev/genai/go/tyk/gateway/proxy_muxer.go:104 +0x3a4
golang.org/x/net/http2/h2c.h2cHandler.ServeHTTP({{0x1062ffc28, 0x140002dcee0}, 0x14000644780}, {0x10630e7b0, 0x140000f2000}, 0x140006147e0)
        /Users/alexander.chernov/go/pkg/mod/golang.org/x/net@v0.29.0/http2/h2c/h2c.go:125 +0xb80
github.com/TykTechnologies/tyk/gateway.(*h2cWrapper).ServeHTTP(0x140004026d8, {0x10630e7b0, 0x140000f2000}, 0x140006147e0)
        /Users/alexander.chernov/dev/genai/go/tyk/gateway/proxy_muxer.go:46 +0x40
net/http.serverHandler.ServeHTTP({0x14000472b40}, {0x10630e7b0, 0x140000f2000}, 0x140006147e0)
        /Users/alexander.chernov/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.6.darwin-arm64/src/net/http/server.go:3142 +0x2b0
net/http.(*conn).serve(0x140001315f0, {0x106311f20, 0x14000f22550})
        /Users/alexander.chernov/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.6.darwin-arm64/src/net/http/server.go:2044 +0x15f8
created by net/http.(*Server).Serve in goroutine 80
        /Users/alexander.chernov/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.6.darwin-arm64/src/net/http/server.go:3290 +0x88c

As per previous report this is caused by the line responseContent, err := io.ReadAll(responseCopy.Body) where responseCopy.Body is nil.

Happening on the 5.3.4 / release-5.3 branch

JanMa commented 1 month ago

@alekc I am running in the same issue. It seems this is caused by enabling opentelemetry support in the Tyk gateway. If you disable it, SSE will work again.