search

polarismesh / polaris

Service Discovery and Governance Platform for Microservice and Distributed Architecture
https://polarismesh.cn
Other
2.41k stars 399 forks source link

ReportClient goroutine leak #1355

Closed yuyongbo closed 5 months ago

yuyongbo commented 5 months ago

Describe the bug

pprof分析

/debug/pprof/goroutine?debug=1 总共77181 协程，grpc的callback_serializer文件 占了67642，协程 未正常退出

goroutine profile: total 77181
67642 @ 0x43d016 0x44cefc 0xad95ea 0x46e7e1
#   0xad95e9    google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run+0x129    /vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:83

具体调用堆栈

6: 576 [6: 576] @ 0xad9175 0xbb3c0f 0xbb849b 0xbb7638 0xf299b1 0xe53a03 0xe5471b 0xe555d3 0xe55838 0xe5573b 0xf2e0ac 0xe6d082 0xe940a9 0xe901b7 0xe9046f 0xe905bf 0x46e7e1
#   0xad9174    google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer+0x34                     vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:52
#   0xbb3c0e    google.golang.org/grpc.newCCBalancerWrapper+0x4e                                vendor/google.golang.org/grpc/balancer_conn_wrappers.go:86
#   0xbb849a    google.golang.org/grpc.(*ClientConn).exitIdleMode+0x2da                             vendor/google.golang.org/grpc/clientconn.go:363
#   0xbb7637    google.golang.org/grpc.DialContext+0x937                                    vendor/google.golang.org/grpc/clientconn.go:261
#   0xf299b0    polaris-go/plugin/serverconnector/grpc.(*Connector).CreateConnection+0x570  vendor/polaris-go/plugin/serverconnector/grpc/creator.go:48
#   0xe53a02    polaris-go/pkg/network.(*ServerAddressList).connectServer+0x222     vendor/polaris-go/pkg/network/impl.go:147
#   0xe5471a    polaris-go/pkg/network.(*ServerAddressList).tryGetConnection+0x25a      vendor/polaris-go/pkg/network/impl.go:222
#   0xe555d2    polaris-go/pkg/network.(*connectionManager).tryGetConnection+0xd2       vendor/polaris-go/pkg/network/impl.go:364
#   0xe55837    polaris-go/pkg/network.(*connectionManager).GetConnectionByHashKey+0x97 vendor/polaris-go/pkg/network/impl.go:376
#   0xe5573a    polaris-go/pkg/network.(*connectionManager).GetConnection+0x5a          vendor/polaris-go/pkg/network/impl.go:369
#   0xf2e0ab    polaris-go/plugin/serverconnector/grpc.(*Connector).ReportClient+0xcb       vendor/polaris-go/plugin/serverconnector/grpc/operation_sync.go:260
#   0xe6d081    polaris-go/pkg/plugin/serverconnector.(*Proxy).ReportClient+0x21        vendor/polaris-go/pkg/plugin/serverconnector/proxy.go:64
#   0xe940a8    polaris-go/pkg/flow/startup.(*ReportClientCallBack).Process+0x188       vendor/polaris-go/pkg/flow/startup/client_report.go:133
#   0xe901b6    polaris-go/pkg/flow/schedule.(*taskRoutine).processPeriodicTask+0x76        vendor/polaris-go/pkg/flow/schedule/routines.go:148
#   0xe9046e    polaris-go/pkg/flow/schedule.(*taskRoutine).iteratePeriodTaskItems+0xee vendor/polaris-go/pkg/flow/schedule/routines.go:169
#   0xe905be    polaris-go/pkg/flow/schedule.(*taskRoutine).runPeriod+0x9e          vendor/polaris-go/pkg/flow/schedule/routines.go:184

Environment