aws / aws-xray-sdk-go

AWS X-Ray SDK for the Go programming language.
Apache License 2.0
276 stars 117 forks source link

goroutine leaks from BeginSegment without context canceling #343

Open momotaro98 opened 2 years ago

momotaro98 commented 2 years ago

Related issue and PR

51 This issue seems to be the original.

156 This PR seemed to have tried to fix the issue but it does't looks so.

How to reproduce goroutine leaks

I've used uber-go/goleak to check if the code makes goroutine leaks.

When an existing test function gets inserted goleak, BeginSegment with context canceling doesn't go to leak.

However, it does when I remove the context canceling like following.

func TestSegmentDataRace(t *testing.T) {
    defer goleak.VerifyNone(t)

    ctx, _ := NewTestDaemon()
    // defer td.Close()
    ctx, _ = context.WithCancel(ctx)
    // defer cancel()

    var wg sync.WaitGroup
    n := 100
    wg.Add(n)
    for i := 0; i < n; i++ { // flaky data race test, so we run it multiple times
        _, seg := BeginSegment(ctx, "TestSegment")

        go func() {
            defer wg.Done()
            seg.Close(nil)
        }()
    }
    // cancel()
    wg.Wait()
}

The result shows errors of goleak

go test ./xray... -run TestSegmentDataRace
2021-12-19T00:12:34+09:00 [INFO] Emitter using address: 127.0.0.1:52731
--- FAIL: TestSegmentDataRace (0.46s)
    leaks.go:78: found unexpected goroutines:
        [Goroutine 36 in state IO wait, with internal/poll.runtime_pollWait on top of the stack:
        goroutine 36 [IO wait]:
        internal/poll.runtime_pollWait(0x98adb18, 0x72)
                /usr/local/go/src/runtime/netpoll.go:229 +0x89
        internal/poll.(*pollDesc).wait(0xc00017c780, 0xc000300000, 0x0)
                /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32
        internal/poll.(*pollDesc).waitRead(...)
                /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
        internal/poll.(*FD).ReadFrom(0xc00017c780, {0xc000300000, 0x10000, 0x10000})
                /usr/local/go/src/internal/poll/fd_unix.go:223 +0x238
        net.(*netFD).readFrom(0xc00017c780, {0xc000300000, 0x0, 0x850538c000010001})
                /usr/local/go/src/net/fd_posix.go:62 +0x29
        net.(*UDPConn).readFrom(0xc000088000, {0xc000300000, 0x0, 0x9840448}, 0xc0000e0ab0)
                /usr/local/go/src/net/udpsock_posix.go:47 +0x3e
        net.(*UDPConn).readFromUDP(0xc00011a3d0, {0xc000300000, 0xc000100660, 0x18}, 0x17fd080)
                /usr/local/go/src/net/udpsock.go:116 +0x31
        net.(*UDPConn).ReadFrom(0xc0000d5f50, {0xc000300000, 0x10000, 0x10000})
                /usr/local/go/src/net/udpsock.go:125 +0x50
        github.com/aws/aws-xray-sdk-go/xray.(*TestDaemon).run(0xc000163a40, 0xc00011d0e0)
                /Users/shintaro/workspace/github.com/momotaro98/aws-xray-sdk-go/xray/util_test.go:82 +0x7a
        created by github.com/aws/aws-xray-sdk-go/xray.NewTestDaemon
                /Users/shintaro/workspace/github.com/momotaro98/aws-xray-sdk-go/xray/util_test.go:55 +0x370

         Goroutine 37 in state chan receive, with github.com/aws/aws-xray-sdk-go/xray.BeginSegmentWithSampling.func1 on top of the stack:
        goroutine 37 [chan receive]:
        github.com/aws/aws-xray-sdk-go/xray.BeginSegmentWithSampling.func1()
                /Users/shintaro/workspace/github.com/momotaro98/aws-xray-sdk-go/xray/segment.go:136 +0x32
        created by github.com/aws/aws-xray-sdk-go/xray.BeginSegmentWithSampling
                /Users/shintaro/workspace/github.com/momotaro98/aws-xray-sdk-go/xray/segment.go:135 +0xf35
momotaro98 commented 2 years ago

I realized there's already a PR #327 for this issue.