open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
3.03k stars 2.34k forks source link

[exporter/elasticsearch] goroutine leaks in tests #35638

Open mauri870 opened 2 weeks ago

mauri870 commented 2 weeks ago

Component(s)

exporter/elasticsearch

Describe the issue you're reporting

Description

The goleak tool reports goroutine leaks in tests.

Steps to Reproduce

cd exporter/elasticsearchexporter
go test ./... -v

Expected Result

No leaks reported by goleak.

Actual Result

goleak: Errors on successful test run: found unexpected goroutines:
[Goroutine 765 in state select, with net/http.(*persistConn).writeLoop on top of the stack:
net/http.(*persistConn).writeLoop(0xc000485320)
        /home/mauri870/gopath/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.1.linux-amd64/src/net/http/transport.go:2519 +0xe7
created by net/http.(*Transport).dialConn in goroutine 823
        /home/mauri870/gopath/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.1.linux-amd64/src/net/http/transport.go:1875 +0x15a5
 Goroutine 764 in state IO wait, with internal/poll.runtime_pollWait on top of the stack:
internal/poll.runtime_pollWait(0x70ffb101fb60, 0x72)
        /home/mauri870/gopath/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.1.linux-amd64/src/runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc000418d80?, 0xc000590000?, 0x0)
        /home/mauri870/gopath/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.1.linux-amd64/src/internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
        /home/mauri870/gopath/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.1.linux-amd64/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000418d80, {0xc000590000, 0x1000, 0x1000})

[...]

Related issues

github-actions[bot] commented 2 weeks ago

Pinging code owners:

mauri870 commented 2 weeks ago

As of now there are two leaks that are detected, following the goleak docs I was able to pinpoint the exact tests that have leaks:

$ export GOTOOLCHAIN=go1.23.1
$ export GOSUMDB=sum.golang.org

$ cd exporter/elasticsearchexporter
$ go test -c -o tests; for test in $(go test -list . | grep -E "^(Test|Example)"); do ./tests -test.run "^$test\$" &>/dev/null && echo -n "." || echo -e "\n$test failed"; done

.......................
TestComponentLifecycle failed
................%

I was unable to fix this particular test, but commenting it reports TestExporterMetrics instead, which I fixed at https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/35639. After that no more leaks are reported.

I would appreciate it if someone could take a look at TestComponentLifecycle.

carsonip commented 2 weeks ago

/label -needs-triage

mauri870 commented 2 weeks ago

Turns out TestExporterMetrics was being reported because of a flaky assertion. We should probably fix TestComponentLifecycle first and make sure all the other tests are not being reported by goleak.

carsonip commented 1 week ago

I have found another leak of unclosed bulk indexer, fixing in https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/35865/commits/210306ef6885f6f4fb88b3b31f759397acfae2fb as part of #35865