search

go-kratos / kratos

Your ultimate Go microservices framework for the cloud-native era.
https://go-kratos.dev
MIT License
23.06k stars 3.97k forks source link

使用北极星作为服务发现时，在注册多个grpc client时，启动报错：concurrent map iteration and map write #3361

Closed JackChanw closed 1 week ago

JackChanw commented 1 month ago

What happened:

INFO msg=watcher's ctx cancel : context canceled INFO msg=[resolver] update instances: [{"id":"7e55bf0237ad7a213c338824f08a89552fa2a1fb","name":"bv_media_server.grpc","version":"","metadata":{"env":"prod","env_id":"0","env_name":"","env_tag":"","protocol":"grpc","version":"","weight":"100"},"endpoints":["grpc://golden-us04.domob.cn:9789"]},{"id":"d742c26c247bbc377f8a99ac9c10e402f04da209","name":"bv_media_server.grpc","version":"","metadata":{"env":"prod","env_id":"0","env_name":"","env_tag":"","protocol":"grpc","version":"","weight":"100"},"endpoints":["grpc://golden-us03.domob.cn:9789"]}] INFO msg=[resolver] update instances: [{"id":"7e55bf0237ad7a213c338824f08a89552fa2a1fb","name":"bv_media_server.grpc","version":"","metadata":{"env":"prod","env_id":"0","env_name":"","env_tag":"","protocol":"grpc","version":"","weight":"100"},"endpoints":["grpc://golden-us04.domob.cn:9789"]},{"id":"d742c26c247bbc377f8a99ac9c10e402f04da209","name":"bv_media_server.grpc","version":"","metadata":{"env":"prod","env_id":"0","env_name":"","env_tag":"","protocol":"grpc","version":"","weight":"100"},"endpoints":["grpc://golden-us03.domob.cn:9789"]}] fatal error: concurrent map iteration and map write

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

goroutine 170 [running]: github.com/go-kratos/kratos/v2/registry.(ServiceInstance).Equal(0xc00089f630, {0x1636220?, 0xc00089f630}) /home/gitlab-runner/go/pkg/mod/github.com/go-kratos/kratos/v2@v2.7.3/registry/registry.go:88 +0x225 google.golang.org/grpc/attributes.(Attributes).Equal(0x16092a0?, 0xc00067c260) /home/gitlab-runner/go/pkg/mod/google.golang.org/grpc@v1.63.2/attributes/attributes.go:96 +0x1b1 google.golang.org/grpc/resolver.addressMapEntryList.find(...) /home/gitlab-runner/go/pkg/mod/google.golang.org/grpc@v1.63.2/resolver/map.go:64 google.golang.org/grpc/resolver.(AddressMap).Get(0xc000b5d9d8?, {{0xc000028047, 0x19}, {0xc00067a138, 0x12}, 0xc00067c260, 0x0, {0x0, 0x0}}) /home/gitlab-runner/go/pkg/mod/google.golang.org/grpc@v1.63.2/resolver/map.go:75 +0x185 google.golang.org/grpc/balancer/base.(baseBalancer).UpdateClientConnState(0xc000c91f00, {{{0xc000c91e80, 0x2, 0x2}, {0xc00093e9c0, 0x2, 0x2}, 0x0, 0x0}, {0x0, ...}}) /home/gitlab-runner/go/pkg/mod/google.golang.org/grpc@v1.63.2/balancer/base/balancer.go:125 +0x4a5 google.golang.org/grpc/internal/balancer/gracefulswitch.(*Balancer).UpdateClientConnState(0xffffffffffffffff?, {{{0xc000c91e80, 0x2

Environment:

JackChanw commented 1 month ago

目前服务为Task模式，启动时使用discovery 链接多个grpc服务，其中有部分服务属于同一个实例（endpoint一致）

kratos-ci-bot commented 1 month ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

The current service is in Task mode. Discovery is used to link multiple grpc services when starting. Some of the services belong to the same instance (the endpoints are consistent).

JackChanw commented 1 month ago

完整错误栈 goroutine 157 [running]: github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery.parseAttributes(...) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery/resolver.go:111 github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery.(discoveryResolver).update(0x140009bfe60, {0x14000af8410, 0xa, 0xa}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery/resolver.go:81 +0x368 github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery.(discoveryResolver).watch(0x140009bfe60) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery/resolver.go:47 +0x74 created by github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery.(*builder).Build /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery/builder.go:116 +0x414

goroutine 1 [runnable]: google.golang.org/grpc.(ccResolverWrapper).start(0x14000a16980) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/resolver_wrapper.go:84 +0xc4 google.golang.org/grpc.(ClientConn).exitIdleMode(0x14000afc400) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/clientconn.go:328 +0x50 google.golang.org/grpc.(idler).ExitIdleMode(0x0?) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/clientconn.go:311 +0x1c google.golang.org/grpc/internal/idle.(Manager).ExitIdleMode(0x140000b8480) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/idle/idle.go:236 +0xb8 google.golang.org/grpc.DialContext({0x105bdf8e8, 0x14000134050}, {0x14000802220, 0x1f}, {0x140009bfec0, 0x5, 0x0?}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/clientconn.go:231 +0x1a0 github.com/go-kratos/kratos/v2/transport/grpc.dial({0x105bdf8e8, 0x14000134050}, 0x1, {0x14000667b48, 0x4, 0x1050d6fac?}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-kratos/kratos/v2/transport/grpc/client.go:204 +0xa90 github.com/go-kratos/kratos/v2/transport/grpc.DialInsecure(...) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-kratos/kratos/v2/transport/grpc/client.go:151 git.domob-inc.cn/bluevision/bv_commons/gkratos_x/kxgrpc.(*ClientFactory).CreateConnWithTimeout(0x14000667e08, {0x14000802220, 0x1f}, 0x2540be400) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/git.domob-inc.cn/bluevision/bv_commons/gkratos_x/kxgrpc/client_factory.go:48 +0x2c8 bv_order_server/internal/data/client.NewPayCenterLakalaClient(0x14000910a80, 0x1400074f040?) /Users/chanw/Domob/Bluevision/bv_order_server/internal/data/client/grpc_client.go:29 +0x60 main.initApp(0x140002db900, 0x14000910a80, {0x105bcbd00, 0x1400074f040}) /Users/chanw/Domob/Bluevision/bv_order_server/cmd/bv_order_server/wire_gen.go:119 +0x4f4 main.main() /Users/chanw/Domob/Bluevision/bv_order_server/cmd/bv_order_server/main.go:58 +0x14c

goroutine 36 [chan receive]: github.com/polarismesh/polaris-go/pkg/clock.(*clockImpl).updateTime(0x140003e37f0) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/pkg/clock/clock.go:57 +0x88 created by github.com/polarismesh/polaris-go/pkg/clock.init.0 /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/pkg/clock/clock.go:74 +0x120

goroutine 10 [select]: github.com/patrickmn/go-cache.(*janitor).Run(0x14000624c60, 0x0?) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/patrickmn/go-cache/cache.go:1079 +0x78 created by github.com/patrickmn/go-cache.runJanitor /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/patrickmn/go-cache/cache.go:1099 +0x100

goroutine 24 [select]: github.com/polarismesh/polaris-go/plugin/metrics/prometheus.(PullAction).doAggregation(0x140002a40f0, {0x105bdf8b0, 0x14000258380}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/metrics/prometheus/reporter.go:299 +0xa4 created by github.com/polarismesh/polaris-go/plugin/metrics/prometheus.(PullAction).Run /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/metrics/prometheus/reporter.go:312 +0xa4

goroutine 26 [chan receive]: github.com/natefinch/lumberjack.(Logger).millRun(0x14000132720) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/natefinch/lumberjack/lumberjack.go:379 +0x44 created by github.com/natefinch/lumberjack.(Logger).mill.func1 /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/natefinch/lumberjack/lumberjack.go:390 +0xa4

goroutine 14 [chan receive]: github.com/natefinch/lumberjack.(Logger).millRun(0x14000132300) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/natefinch/lumberjack/lumberjack.go:379 +0x44 created by github.com/natefinch/lumberjack.(Logger).mill.func1 /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/natefinch/lumberjack/lumberjack.go:390 +0xa4

goroutine 50 [select]: github.com/polarismesh/polaris-go/pkg/network.(*connectionManager).doSwitchRoutine(0x14000160a80) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/pkg/network/impl.go:493 +0xe4 created by github.com/polarismesh/polaris-go/pkg/network.NewConnectionManager /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/pkg/network/impl.go:307 +0x590

goroutine 72 [select]: github.com/polarismesh/polaris-go/plugin/serverconnector/common.(DiscoverConnector).doSend(0x1400022a780) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/serverconnector/common/discover.go:216 +0xc8 created by github.com/polarismesh/polaris-go/plugin/serverconnector/common.(DiscoverConnector).StartUpdateRoutines /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/serverconnector/common/discover.go:123 +0x124

goroutine 73 [select]: github.com/polarismesh/polaris-go/plugin/serverconnector/common.(DiscoverConnector).doRetry(0x1400022a780) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/serverconnector/common/discover.go:171 +0xd0 created by github.com/polarismesh/polaris-go/plugin/serverconnector/common.(DiscoverConnector).StartUpdateRoutines /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/serverconnector/common/discover.go:124 +0x168

goroutine 74 [select]: github.com/polarismesh/polaris-go/plugin/serverconnector/common.(DiscoverConnector).doLog(0x1400022a780) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/serverconnector/common/discover.go:142 +0xd4 created by github.com/polarismesh/polaris-go/plugin/serverconnector/common.(DiscoverConnector).StartUpdateRoutines /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/serverconnector/common/discover.go:125 +0x1ac

goroutine 75 [select]: github.com/polarismesh/polaris-go/plugin/localregistry/inmemory.(LocalCache).logServiceMap(0x1400020a0f0) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/localregistry/inmemory/inmemory.go:195 +0xd4 created by github.com/polarismesh/polaris-go/plugin/localregistry/inmemory.(LocalCache).Start /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/localregistry/inmemory/inmemory.go:220 +0xb4

goroutine 76 [select]: github.com/polarismesh/polaris-go/pkg/flow/schedule.(taskRoutine).runPeriod(0x14000808be0) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/pkg/flow/schedule/routines.go:179 +0xc4 created by github.com/polarismesh/polaris-go/pkg/flow/schedule.(taskRoutine).start /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/pkg/flow/schedule/routines.go:109 +0x2e8

goroutine 77 [select]: github.com/polarismesh/polaris-go/pkg/flow/schedule.(taskRoutine).runPeriod(0x14000808c80) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/pkg/flow/schedule/routines.go:179 +0xc4 created by github.com/polarismesh/polaris-go/pkg/flow/schedule.(taskRoutine).start /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/pkg/flow/schedule/routines.go:109 +0x2e8

goroutine 25 [IO wait]: internal/poll.runtime_pollWait(0x12e8220d8, 0x72) /Users/chanw/Go/go1.19.1/src/runtime/netpoll.go:305 +0xa0 internal/poll.(pollDesc).wait(0x14000796000?, 0x105a4a420?, 0x0) /Users/chanw/Go/go1.19.1/src/internal/poll/fd_poll_runtime.go:84 +0x28 internal/poll.(pollDesc).waitRead(...) /Users/chanw/Go/go1.19.1/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(FD).Accept(0x14000796000) /Users/chanw/Go/go1.19.1/src/internal/poll/fd_unix.go:614 +0x1d0 net.(netFD).accept(0x14000796000) /Users/chanw/Go/go1.19.1/src/net/fd_unix.go:172 +0x28 net.(TCPListener).accept(0x14000282078) /Users/chanw/Go/go1.19.1/src/net/tcpsock_posix.go:142 +0x28 net.(TCPListener).Accept(0x14000282078) /Users/chanw/Go/go1.19.1/src/net/tcpsock.go:288 +0x2c net/http.(Server).Serve(0x1400079c000, {0x105bde510, 0x14000282078}) /Users/chanw/Go/go1.19.1/src/net/http/server.go:3070 +0x30c net/http.Serve(...) /Users/chanw/Go/go1.19.1/src/net/http/server.go:2568 github.com/polarismesh/polaris-go/plugin/metrics/prometheus.(PullAction).Run.func1() /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/metrics/prometheus/reporter.go:327 +0x354 created by github.com/polarismesh/polaris-go/plugin/metrics/prometheus.(*PullAction).Run /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/metrics/prometheus/reporter.go:313 +0xe8

goroutine 44 [chan receive]: github.com/natefinch/lumberjack.(Logger).millRun(0x14000132660) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/natefinch/lumberjack/lumberjack.go:379 +0x44 created by github.com/natefinch/lumberjack.(Logger).mill.func1 /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/natefinch/lumberjack/lumberjack.go:390 +0xa4

goroutine 147 [select]: github.com/go-sql-driver/mysql.(mysqlConn).startWatcher.func1() /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-sql-driver/mysql/connection.go:614 +0x7c created by github.com/go-sql-driver/mysql.(mysqlConn).startWatcher /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-sql-driver/mysql/connection.go:611 +0x110

goroutine 106 [select]: github.com/polarismesh/polaris-go/plugin/serverconnector/common.(DiscoverConnector).doSend(0x140004e0180) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/serverconnector/common/discover.go:216 +0xc8 created by github.com/polarismesh/polaris-go/plugin/serverconnector/common.(DiscoverConnector).StartUpdateRoutines /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/serverconnector/common/discover.go:123 +0x124

goroutine 107 [select]: github.com/polarismesh/polaris-go/plugin/serverconnector/common.(DiscoverConnector).doRetry(0x140004e0180) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/serverconnector/common/discover.go:171 +0xd0 created by github.com/polarismesh/polaris-go/plugin/serverconnector/common.(DiscoverConnector).StartUpdateRoutines /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/serverconnector/common/discover.go:124 +0x168

goroutine 108 [select]: github.com/polarismesh/polaris-go/plugin/serverconnector/common.(DiscoverConnector).doLog(0x140004e0180) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/serverconnector/common/discover.go:142 +0xd4 created by github.com/polarismesh/polaris-go/plugin/serverconnector/common.(DiscoverConnector).StartUpdateRoutines /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/serverconnector/common/discover.go:125 +0x1ac

goroutine 48 [select]: database/sql.(*DB).connectionOpener(0x140004d2ea0, {0x105bdf8b0, 0x14000274040}) /Users/chanw/Go/go1.19.1/src/database/sql/sql.go:1224 +0x80 created by database/sql.OpenDB /Users/chanw/Go/go1.19.1/src/database/sql/sql.go:792 +0x180

goroutine 105 [select]: github.com/polarismesh/polaris-go/plugin/localregistry/inmemory.(LocalCache).logServiceMap(0x14000a02000) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/localregistry/inmemory/inmemory.go:195 +0xd4 created by github.com/polarismesh/polaris-go/plugin/localregistry/inmemory.(LocalCache).Start /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/localregistry/inmemory/inmemory.go:220 +0xb4

goroutine 82 [select]: github.com/polarismesh/polaris-go/pkg/network.(*connectionManager).doSwitchRoutine(0x140004cc000) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/pkg/network/impl.go:493 +0xe4 created by github.com/polarismesh/polaris-go/pkg/network.NewConnectionManager /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/pkg/network/impl.go:307 +0x590

goroutine 109 [select]: github.com/polarismesh/polaris-go/pkg/flow/schedule.(taskRoutine).runPeriod(0x140002a5090) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/pkg/flow/schedule/routines.go:179 +0xc4 created by github.com/polarismesh/polaris-go/pkg/flow/schedule.(taskRoutine).start /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/pkg/flow/schedule/routines.go:109 +0x2e8

goroutine 110 [select]: github.com/polarismesh/polaris-go/pkg/flow/schedule.(taskRoutine).runPeriod(0x140002a5130) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/pkg/flow/schedule/routines.go:179 +0xc4 created by github.com/polarismesh/polaris-go/pkg/flow/schedule.(taskRoutine).start /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/pkg/flow/schedule/routines.go:109 +0x2e8

goroutine 111 [select]: github.com/polarismesh/polaris-go/plugin/metrics/prometheus.(PullAction).doAggregation(0x140002a5220, {0x105bdf8b0, 0x14000258d00}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/metrics/prometheus/reporter.go:299 +0xa4 created by github.com/polarismesh/polaris-go/plugin/metrics/prometheus.(PullAction).Run /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/metrics/prometheus/reporter.go:312 +0xa4

goroutine 137 [IO wait]: internal/poll.runtime_pollWait(0x12e821fe8, 0x72) /Users/chanw/Go/go1.19.1/src/runtime/netpoll.go:305 +0xa0 internal/poll.(pollDesc).wait(0x14000796780?, 0x105a4a420?, 0x0) /Users/chanw/Go/go1.19.1/src/internal/poll/fd_poll_runtime.go:84 +0x28 internal/poll.(pollDesc).waitRead(...) /Users/chanw/Go/go1.19.1/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(FD).Accept(0x14000796780) /Users/chanw/Go/go1.19.1/src/internal/poll/fd_unix.go:614 +0x1d0 net.(netFD).accept(0x14000796780) /Users/chanw/Go/go1.19.1/src/net/fd_unix.go:172 +0x28 net.(TCPListener).accept(0x14000283008) /Users/chanw/Go/go1.19.1/src/net/tcpsock_posix.go:142 +0x28 net.(TCPListener).Accept(0x14000283008) /Users/chanw/Go/go1.19.1/src/net/tcpsock.go:288 +0x2c net/http.(Server).Serve(0x1400079c3c0, {0x105bde510, 0x14000283008}) /Users/chanw/Go/go1.19.1/src/net/http/server.go:3070 +0x30c net/http.(Server).ListenAndServe(0x1400079c3c0) /Users/chanw/Go/go1.19.1/src/net/http/server.go:2999 +0x84 net/http.ListenAndServe({0x1400049d678, 0x6}, {0x0?, 0x0}) /Users/chanw/Go/go1.19.1/src/net/http/server.go:3255 +0x94 created by git.domob-inc.cn/bluefocus/golang_libs/bfomonitor.start /Users/chanw/Domob/Bluevision/bv_order_server/vendor/git.domob-inc.cn/bluefocus/golang_libs/bfomonitor/pmonitor.go:111 +0x1d4

goroutine 140 [select]: go.opentelemetry.io/otel/sdk/trace.(*batchSpanProcessor).processQueue(0x140006f94a0) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/go.opentelemetry.io/otel/sdk/trace/batch_span_processor.go:293 +0xd8 go.opentelemetry.io/otel/sdk/trace.NewBatchSpanProcessor.func1() /Users/chanw/Domob/Bluevision/bv_order_server/vendor/go.opentelemetry.io/otel/sdk/trace/batch_span_processor.go:128 +0x58 created by go.opentelemetry.io/otel/sdk/trace.NewBatchSpanProcessor /Users/chanw/Domob/Bluevision/bv_order_server/vendor/go.opentelemetry.io/otel/sdk/trace/batch_span_processor.go:126 +0x2a4

goroutine 141 [select]: database/sql.(*DB).connectionOpener(0x140001d8340, {0x105bdf8b0, 0x1400064d9c0}) /Users/chanw/Go/go1.19.1/src/database/sql/sql.go:1224 +0x80 created by database/sql.OpenDB /Users/chanw/Go/go1.19.1/src/database/sql/sql.go:792 +0x180

goroutine 144 [select]: github.com/go-sql-driver/mysql.(mysqlConn).startWatcher.func1() /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-sql-driver/mysql/connection.go:614 +0x7c created by github.com/go-sql-driver/mysql.(mysqlConn).startWatcher /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-sql-driver/mysql/connection.go:611 +0x110

goroutine 166 [select]: google.golang.org/grpc/internal/transport.(recvBufferReader).readClient(0x140006dd720, {0x14000431480, 0x5, 0x5}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/transport/transport.go:193 +0x74 google.golang.org/grpc/internal/transport.(recvBufferReader).Read(0x140006dd720, {0x14000431480?, 0x140005fa3d8?, 0x14000987740?}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/transport/transport.go:173 +0x148 google.golang.org/grpc/internal/transport.(transportReader).Read(0x14000431230, {0x14000431480?, 0x140009877c8?, 0x104b900b8?}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/transport/transport.go:525 +0x34 io.ReadAtLeast({0x105bce540, 0x14000431230}, {0x14000431480, 0x5, 0x5}, 0x5) /Users/chanw/Go/go1.19.1/src/io/io.go:332 +0xa0 io.ReadFull(...) /Users/chanw/Go/go1.19.1/src/io/io.go:351 google.golang.org/grpc/internal/transport.(Stream).Read(0x14000922c60, {0x14000431480, 0x5, 0x5}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/transport/transport.go:509 +0x98 google.golang.org/grpc.(parser).recvMsg(0x14000431470, 0x3200000) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/rpc_util.go:614 +0x48 google.golang.org/grpc.recvAndDecompress(0x14000431470, 0x14000922c60, {0x0, 0x0}, 0x3200000, 0x0, {0x0, 0x0}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/rpc_util.go:753 +0x58 google.golang.org/grpc.recv(0x140009879f8?, {0x12e82c508, 0x1068bd750}, 0x1045f3e18?, {0x0?, 0x0?}, {0x105b4a4c0, 0x1400022b380}, 0x65?, 0x0?, ...) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/rpc_util.go:833 +0x64 google.golang.org/grpc.(csAttempt).recvMsg(0x140001d85b0, {0x105b4a4c0?, 0x1400022b380}, 0x140008bf6d8?) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/stream.go:1085 +0x240 google.golang.org/grpc.(clientStream).RecvMsg.func1(0xd900000106e8ca68?) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/stream.go:928 +0x28 google.golang.org/grpc.(clientStream).withRetry(0x14000922a20, 0x14000987c38, 0x14000987c28) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/stream.go:761 +0x568 google.golang.org/grpc.(clientStream).RecvMsg(0x14000922a20, {0x105b4a4c0?, 0x1400022b380?}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/stream.go:927 +0xbc github.com/polarismesh/specification/source/go/api/v1/service_manage.(polarisGRPCDiscoverClient).Recv(0x14000737850) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/specification/source/go/api/v1/service_manage/grpcapi.pb.go:110 +0x58 github.com/polarismesh/polaris-go/plugin/serverconnector/common.(StreamingClient).receiveAndNotify(0x140001b2230) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/serverconnector/common/discover.go:479 +0x138 created by github.com/polarismesh/polaris-go/plugin/serverconnector/common.(DiscoverConnector).newStream /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/plugin/serverconnector/common/discover.go:604 +0x828

goroutine 139 [IO wait]: internal/poll.runtime_pollWait(0x12e821ef8, 0x72) /Users/chanw/Go/go1.19.1/src/runtime/netpoll.go:305 +0xa0 internal/poll.(pollDesc).wait(0x140000ce000?, 0x105a4a420?, 0x0) /Users/chanw/Go/go1.19.1/src/internal/poll/fd_poll_runtime.go:84 +0x28 internal/poll.(pollDesc).waitRead(...) /Users/chanw/Go/go1.19.1/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(FD).Accept(0x140000ce000) /Users/chanw/Go/go1.19.1/src/internal/poll/fd_unix.go:614 +0x1d0 net.(netFD).accept(0x140000ce000) /Users/chanw/Go/go1.19.1/src/net/fd_unix.go:172 +0x28 net.(TCPListener).accept(0x14000a00288) /Users/chanw/Go/go1.19.1/src/net/tcpsock_posix.go:142 +0x28 net.(TCPListener).Accept(0x14000a00288) /Users/chanw/Go/go1.19.1/src/net/tcpsock.go:288 +0x2c net/http.(Server).Serve(0x14000a020f0, {0x105bde510, 0x14000a00288}) /Users/chanw/Go/go1.19.1/src/net/http/server.go:3070 +0x30c net/http.(Server).ListenAndServe(0x14000a020f0) /Users/chanw/Go/go1.19.1/src/net/http/server.go:2999 +0x84 net/http.ListenAndServe(...) /Users/chanw/Go/go1.19.1/src/net/http/server.go:3255 git.domob-inc.cn/bluevision/bv_commons/gkratos_x/kxserverob.InitServerObserver.func1() /Users/chanw/Domob/Bluevision/bv_order_server/vendor/git.domob-inc.cn/bluevision/bv_commons/gkratos_x/kxserverob/server_observer.go:13 +0x40 created by git.domob-inc.cn/bluevision/bv_commons/gkratos_x/kxserverob.InitServerObserver /Users/chanw/Domob/Bluevision/bv_order_server/vendor/git.domob-inc.cn/bluevision/bv_commons/gkratos_x/kxserverob/server_observer.go:12 +0x30

goroutine 47 [select]: database/sql.(DB).connectionCleaner(0x140001d8340, 0x0?) /Users/chanw/Go/go1.19.1/src/database/sql/sql.go:1067 +0x90 created by database/sql.(DB).startCleanerLocked /Users/chanw/Go/go1.19.1/src/database/sql/sql.go:1054 +0x108

goroutine 148 [select]: google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0x140009f0e70, {0x105bdf8b0, 0x14000275500}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:76 +0xc8 created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:52 +0x144

goroutine 118 [select]: github.com/go-sql-driver/mysql.(mysqlConn).startWatcher.func1() /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-sql-driver/mysql/connection.go:614 +0x7c created by github.com/go-sql-driver/mysql.(mysqlConn).startWatcher /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-sql-driver/mysql/connection.go:611 +0x110

goroutine 119 [select]: database/sql.(DB).connectionCleaner(0x140004d2ea0, 0x0?) /Users/chanw/Go/go1.19.1/src/database/sql/sql.go:1067 +0x90 created by database/sql.(DB).startCleanerLocked /Users/chanw/Go/go1.19.1/src/database/sql/sql.go:1054 +0x108

goroutine 120 [select]: database/sql.(*DB).connectionOpener(0x140004e4dd0, {0x105bdf8b0, 0x14000259f80}) /Users/chanw/Go/go1.19.1/src/database/sql/sql.go:1224 +0x80 created by database/sql.OpenDB /Users/chanw/Go/go1.19.1/src/database/sql/sql.go:792 +0x180

goroutine 123 [select]: database/sql.(DB).connectionCleaner(0x140004e4dd0, 0x0?) /Users/chanw/Go/go1.19.1/src/database/sql/sql.go:1067 +0x90 created by database/sql.(DB).startCleanerLocked /Users/chanw/Go/go1.19.1/src/database/sql/sql.go:1054 +0x108

goroutine 124 [select]: google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0x14000270580, {0x105bdf8b0, 0x1400074e180}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:76 +0xc8 created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:52 +0x144

goroutine 125 [select]: google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0x140002705b0, {0x105bdf8b0, 0x1400074e200}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:76 +0xc8 created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:52 +0x144

goroutine 126 [select]: google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0x140002705e0, {0x105bdf8b0, 0x1400074e280}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:76 +0xc8 created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:52 +0x144

goroutine 152 [runnable]: encoding/json.stateInString(0x140007d5ec0?, 0x30?) /Users/chanw/Go/go1.19.1/src/encoding/json/scanner.go:337 +0x1d8 encoding/json.Indent(0x14000a3d8b0, {0x14000b0a000, 0x1dee, 0x1045fd370?}, {0x0, 0x0}, {0x10553139a, 0x2}) /Users/chanw/Go/go1.19.1/src/encoding/json/indent.go:89 +0xf8 encoding/json.MarshalIndent({0x105a72940?, 0x140007d5e80?}, {0x0, 0x0}, {0x10553139a, 0x2}) /Users/chanw/Go/go1.19.1/src/encoding/json/encode.go:180 +0x68 google.golang.org/grpc/internal/pretty.ToJSON({0x105a72940, 0x140007d5e80}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/pretty/pretty.go:56 +0x198 google.golang.org/grpc.(ccResolverWrapper).addChannelzTraceEvent(0x14000797280, {{0x1400080c000, 0xa, 0xa}, {0x1400017c780, 0xa, 0xa}, 0x0, 0x0}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/resolver_wrapper.go:197 +0x2e0 google.golang.org/grpc.(ccResolverWrapper).UpdateState(0x14000797280, {{0x1400080c000, 0xa, 0xa}, {0x1400017c780, 0xa, 0xa}, 0x0, 0x0}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/resolver_wrapper.go:133 +0x1b0 github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery.(discoveryResolver).update(0x140009bf7a0, {0x140004e3d60, 0xa, 0xa}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery/resolver.go:90 +0x5e4 github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery.(discoveryResolver).watch(0x140009bf7a0) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery/resolver.go:47 +0x74 created by github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery.(*builder).Build /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery/builder.go:116 +0x414

goroutine 180 [select]: google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0x14000271190, {0x105bdf8b0, 0x1400074f0c0}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:76 +0xc8 created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:52 +0x144

goroutine 179 [runnable]: google.golang.org/grpc/attributes.(Attributes).String(0x14000626990?) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/attributes/attributes.go:113 +0x100 google.golang.org/grpc/attributes.(Attributes).MarshalJSON(0x1059eeee0?) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/attributes/attributes.go:140 +0x1c encoding/json.marshalerEncoder(0x140009d8a80, {0x1059eeee0?, 0x14000656720?, 0x14000783b00?}, {0x0?, 0x10?}) /Users/chanw/Go/go1.19.1/src/encoding/json/encode.go:478 +0xc4 encoding/json.structEncoder.encode({{{0x1400027b680?, 0x5?, 0x8?}, 0x140008fb230?}}, 0x140009d8a80, {0x105ad6a80?, 0x14000656700?, 0x1047457d4?}, {0x0?, 0x1?}) /Users/chanw/Go/go1.19.1/src/encoding/json/encode.go:760 +0x178 encoding/json.arrayEncoder.encode({0x0?}, 0x140009d8a80, {0x10587b380?, 0x140008fc2c0?, 0x0?}, {0x0?, 0x0?}) /Users/chanw/Go/go1.19.1/src/encoding/json/encode.go:915 +0xc0 encoding/json.sliceEncoder.encode({0x1400066d608?}, 0x140009d8a80, {0x10587b380?, 0x140008fc2c0?, 0x2600000000000000?}, {0xc?, 0x0?}) /Users/chanw/Go/go1.19.1/src/encoding/json/encode.go:888 +0x298 encoding/json.structEncoder.encode({{{0x140007898c0?, 0x105bf41d8?, 0x0?}, 0x140008fb410?}}, 0x140009d8a80, {0x105a72940?, 0x140008fc2c0?, 0x105a72940?}, {0x48?, 0xd7?}) /Users/chanw/Go/go1.19.1/src/encoding/json/encode.go:760 +0x178 encoding/json.(encodeState).reflectValue(0x1400066d7c8?, {0x105a72940?, 0x140008fc2c0?, 0x1400066d818?}, {0x2?, 0x0?}) /Users/chanw/Go/go1.19.1/src/encoding/json/encode.go:359 +0x70 encoding/json.(encodeState).marshal(0x1071ebfff?, {0x105a72940?, 0x140008fc2c0?}, {0x0?, 0x4?}) /Users/chanw/Go/go1.19.1/src/encoding/json/encode.go:331 +0x120 encoding/json.Marshal({0x105a72940, 0x140008fc2c0}) /Users/chanw/Go/go1.19.1/src/encoding/json/encode.go:160 +0x3c encoding/json.MarshalIndent({0x105a72940?, 0x140008fc2c0?}, {0x0, 0x0}, {0x10553139a, 0x2}) /Users/chanw/Go/go1.19.1/src/encoding/json/encode.go:175 +0x34 google.golang.org/grpc/internal/pretty.ToJSON({0x105a72940, 0x140008fc2c0}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/pretty/pretty.go:56 +0x198 google.golang.org/grpc.(ccResolverWrapper).addChannelzTraceEvent(0x14000a16900, {{0x14000656500, 0xa, 0xa}, {0x14000260b40, 0xa, 0xa}, 0x0, 0x0}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/resolver_wrapper.go:197 +0x2e0 google.golang.org/grpc.(ccResolverWrapper).UpdateState(0x14000a16900, {{0x14000656500, 0xa, 0xa}, {0x14000260b40, 0xa, 0xa}, 0x0, 0x0}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/resolver_wrapper.go:133 +0x1b0 github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery.(discoveryResolver).update(0x140005b3920, {0x14000651d60, 0xa, 0xa}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery/resolver.go:90 +0x5e4 github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery.(discoveryResolver).watch(0x140005b3920) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery/resolver.go:47 +0x74 created by github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery.(*builder).Build /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery/builder.go:116 +0x414

goroutine 149 [select]: google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0x140009f0ea0, {0x105bdf8b0, 0x14000275640}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:76 +0xc8 created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:52 +0x144

goroutine 150 [select]: google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0x140009f0ed0, {0x105bdf8b0, 0x14000275680}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:76 +0xc8 created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:52 +0x144

goroutine 165 [select]: google.golang.org/grpc.newClientStreamWithParams.func4() /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/stream.go:392 +0x8c created by google.golang.org/grpc.newClientStreamWithParams /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/stream.go:391 +0xc44

goroutine 163 [runnable]: internal/poll.runtime_pollWait(0x12e821b38, 0x72) /Users/chanw/Go/go1.19.1/src/runtime/netpoll.go:305 +0xa0 internal/poll.(pollDesc).wait(0x14000909680?, 0x14000630000?, 0x0) /Users/chanw/Go/go1.19.1/src/internal/poll/fd_poll_runtime.go:84 +0x28 internal/poll.(pollDesc).waitRead(...) /Users/chanw/Go/go1.19.1/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(FD).Read(0x14000909680, {0x14000630000, 0x8000, 0x8000}) /Users/chanw/Go/go1.19.1/src/internal/poll/fd_unix.go:167 +0x1e0 net.(netFD).Read(0x14000909680, {0x14000630000?, 0x140008c3ca8?, 0x1046f271c?}) /Users/chanw/Go/go1.19.1/src/net/fd_posix.go:55 +0x28 net.(conn).Read(0x140006265f8, {0x14000630000?, 0x140008c3d18?, 0x104b7d80c?}) /Users/chanw/Go/go1.19.1/src/net/net.go:183 +0x34 bufio.(Reader).Read(0x14000911a40, {0x140004cc3c0, 0x9, 0x0?}) /Users/chanw/Go/go1.19.1/src/bufio/bufio.go:237 +0x1e0 io.ReadAtLeast({0x105bc75a0, 0x14000911a40}, {0x140004cc3c0, 0x9, 0x9}, 0x9) /Users/chanw/Go/go1.19.1/src/io/io.go:332 +0xa0 io.ReadFull(...) /Users/chanw/Go/go1.19.1/src/io/io.go:351 golang.org/x/net/http2.readFrameHeader({0x140004cc3c0?, 0x9?, 0x14000430330?}, {0x105bc75a0?, 0x14000911a40?}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/golang.org/x/net/http2/frame.go:237 +0x58 golang.org/x/net/http2.(Framer).ReadFrame(0x140004cc380) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/golang.org/x/net/http2/frame.go:498 +0x84 google.golang.org/grpc/internal/transport.(http2Client).reader(0x14000788b40, 0x0?) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1602 +0x1b8 created by google.golang.org/grpc/internal/transport.newHTTP2Client /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/transport/http2_client.go:409 +0x15b8

goroutine 164 [select]: google.golang.org/grpc/internal/transport.(controlBuffer).get(0x140006dd540, 0x1) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:418 +0x154 google.golang.org/grpc/internal/transport.(loopyWriter).run(0x1400025f0a0) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:551 +0x7c google.golang.org/grpc/internal/transport.newHTTP2Client.func6() /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/transport/http2_client.go:463 +0x70 created by google.golang.org/grpc/internal/transport.newHTTP2Client /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/transport/http2_client.go:461 +0x1a78

goroutine 27 [select]: github.com/polarismesh/polaris-go/pkg/flow/schedule.(taskRoutine).runTakePriority(0x14000808960) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/pkg/flow/schedule/routines.go:133 +0xd0 created by github.com/polarismesh/polaris-go/pkg/flow/schedule.(taskRoutine).start /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/polarismesh/polaris-go/pkg/flow/schedule/routines.go:106 +0x1b4

goroutine 153 [select]: google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0x140006249e0, {0x105bdf8b0, 0x14000a566c0}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:76 +0xc8 created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:52 +0x144

goroutine 154 [select]: google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0x14000624a20, {0x105bdf8b0, 0x14000a56740}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:76 +0xc8 created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:52 +0x144

goroutine 155 [select]: google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0x14000624a70, {0x105bdf8b0, 0x14000a56780}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:76 +0xc8 created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:52 +0x144

goroutine 181 [select]: google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0x140002711d0, {0x105bdf8b0, 0x1400074f640}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:76 +0xc8 created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:52 +0x144

goroutine 182 [select]: google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0x14000271220, {0x105bdf8b0, 0x1400074f6c0}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:76 +0xc8 created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:52 +0x144

goroutine 158 [select]: google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0x140006254a0, {0x105bdf8b0, 0x14000a568c0}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:76 +0xc8 created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:52 +0x144

goroutine 159 [select]: google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0x140006254e0, {0x105bdf8b0, 0x14000a56940}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:76 +0xc8 created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:52 +0x144

goroutine 160 [select]: google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0x14000625510, {0x105bdf8b0, 0x14000a56980}) /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:76 +0xc8 created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer /Users/chanw/Domob/Bluevision/bv_order_server/vendor/google.golang.org/grpc/internal/grpcsync/callback_serializer.go:52 +0x144

goroutine 194 [runnable]: github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery.(builder).Build.func2() /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery/builder.go:116 runtime.goexit() /Users/chanw/Go/go1.19.1/src/runtime/asm_arm64.s:1172 +0x4 created by github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery.(builder).Build /Users/chanw/Domob/Bluevision/bv_order_server/vendor/github.com/go-kratos/kratos/v2/transport/grpc/resolver/discovery/builder.go:116 +0x414

shenqidebaozi commented 1 month ago

Can you help fix it directly