search

apache / servicecomb-pack

Apache ServiceComb Pack is an eventually data consistency solution for micro-service applications. ServiceComb Pack currently provides TCC and Saga distributed transaction co-ordination solutions by using Alpha as a transaction coordinator and Omega as an transaction agent .
https://servicecomb.apache.org/
Apache License 2.0
1.93k stars 436 forks source link

Omega:Stream 3 sent too many headers EOS: false #738

Open Hakunata opened 2 years ago

Hakunata commented 2 years ago

用jmeter本地压力测试，发现一台服务器报错： `Releasing transactional SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@7f4be458] Transaction synchronization committing SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@7f4be458] Transaction synchronization deregistering SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@7f4be458] Transaction synchronization closing SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@7f4be458] 2022-02-25 14:13:32.420 ERROR 11273 --- [ault-executor-1] o.a.s.p.o.c.g.c.ReconnectStreamObserver : Failed to process grpc coordinate command.

io.grpc.StatusRuntimeException: INTERNAL: HTTP/2 error code: INTERNAL_ERROR Received Goaway Stream 3 sent too many headers EOS: false at io.grpc.Status.asRuntimeException(Status.java:533) ~[grpc-api-1.22.0.jar:1.22.0] at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:442) [grpc-stub-1.22.0.jar:1.22.0] at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39) [grpc-api-1.22.0.jar:1.22.0] at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23) [grpc-api-1.22.0.jar:1.22.0] at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40) [grpc-api-1.22.0.jar:1.22.0] at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:700) [grpc-core-1.22.0.jar:1.22.0] at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39) [grpc-api-1.22.0.jar:1.22.0] at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23) [grpc-api-1.22.0.jar:1.22.0] at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40) [grpc-api-1.22.0.jar:1.22.0] at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:399) [grpc-core-1.22.0.jar:1.22.0] at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:507) [grpc-core-1.22.0.jar:1.22.0] at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:66) [grpc-core-1.22.0.jar:1.22.0] at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:627) [grpc-core-1.22.0.jar:1.22.0] at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$700(ClientCallImpl.java:515) [grpc-core-1.22.0.jar:1.22.0] at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:686) [grpc-core-1.22.0.jar:1.22.0] at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:675) [grpc-core-1.22.0.jar:1.22.0] at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) [grpc-core-1.22.0.jar:1.22.0] at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) [grpc-core-1.22.0.jar:1.22.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_211] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_211] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_211]

2022-02-25 14:13:32.421 INFO 11273 --- [pool-3-thread-1] .a.s.p.o.c.g.c.PushBackReconnectRunnable : Retry connecting to alpha at 127.0.0.1:8080 2022-02-25 14:13:32.438 INFO 11273 --- [pool-3-thread-1] .a.s.p.o.c.g.c.PushBackReconnectRunnable : Retry connecting to alpha at 127.0.0.1:8080 is successful`

WillemJiang commented 2 years ago

压力上到多少？ 从stack trace 来看应该是grpc的问题。 昨天在master上刚刚升级了grpc的版本，你可以试一下。