search

apache / dubbo

The java implementation of Apache Dubbo. An RPC and microservice framework.
https://dubbo.apache.org/
Apache License 2.0
40.53k stars 26.43k forks source link

Same server thread can only throw an exception once, and assigning it again causes the caller to timeout #13688

Closed iwangjie closed 9 months ago

iwangjie commented 10 months ago

Version information

DUBBO： 3.1.11 JDK: 1.8

JVM parameters

-Dlogging.level.org.apache.dubbo=debug -Dlogging.level.org.jboss.netty=debug -Dspring.rabbitmq.listener.allow-start-up=true -Ddubbo.protocol.name=tri -Ddubbo.application.enable-empty-protection=false -Ddubbo.application.serialize-check-status=DISABLE -Ddubbo.provider.prefer-serialization=hessian2 -Dmanagement.endpoints.web.base-path=/actuator -Dmanagement.endpoints.web.exposure.include=*

Phenomenon

Server startup casually throws an exception, the same server thread, the client can receive it for the first time, and the client will time out for the second time. client :Waiting server-side response timeout

Expect

No timeout occurs when returning results normally

Client timeout stack

org.apache.dubbo.rpc.StatusRpcException: DEADLINE_EXCEEDED : Waiting server-side response time......
2024-01-25 14:46:39.862 [system-unknown] [TID: N/A] [NettyClientWorker-7-4] DEBUG o.a.d.remoting.transport.netty4.NettyClientHandler:? -  [DUBBO] IdleStateEvent triggered, send heartbeat to channel NettyChannel [channel=[id: 0x101bc7f1, L:/192.168.48.17:63359 - R:/192.168.48.17:20880]], dubbo version: 3.1.11, current host: 192.168.48.17
2024-01-25 14:46:48.588 [system-unknown] [TID: N/A] [http-nio-9991-exec-5] WARN  o.apache.dubbo.rpc.proxy.InvokerInvocationHandler:? -  [DUBBO] [Dubbo-Consumer] execute service test/com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI#batchSaveEmployee cost 30030.022475 ms, this invocation almost (maybe already) timeout. Timeout: 30000ms
invocation context:
LOGIN_PRINCIPAL={"id":"1685908412536033281","loginAcc":"zhangwuji","name":"zhangwuji","type":"OMS_USER"};
remote.application=sp-oms-consumer;
dubbo-auth-token=8f9dfdf700e645dc9ad08ccd30089b65;
dubbo-auth-key-ver=v2;
dubbo-auth-seed=LTY2Nzc1MjMxNzk5Ng==;
interface=com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI;
timeout=30000;
group=test;
thread info: 
Start time: 81548293605997
+-[ Offset: 0.000000ms; Usage: 30030.022475ms, 100% ] Receive request. Client invoke begin. ServiceKey: test/com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI MethodName:batchSaveEmployee
  +-[ Offset: 0.456341ms; Usage: 30029.309431ms, 99% ] Invoker invoke. Target Address: 192.168.48.17:50051, dubbo version: 3.1.11, current host: 192.168.48.17, error code: 3-4. This may be caused by , go to https://dubbo.apache.org/faq/3/4 to find instructions. 
2024-01-25 14:46:48.589 [system-unknown] [TID: N/A] [http-nio-9991-exec-5] ERROR c.e.z.c.exception.handler.BaseExceptionHandler:? - RPC调用失败,事件编号：167b609d023049deb20628f02f564ba7org.apache.dubbo.rpc.StatusRpcException: DEADLINE_EXCEEDED : Waiting server-side response timeout by scan timer. start time: 2024-01-25 14:46:18.558, end time: 2024-01-25 14:46:48.587, timeout: 30000 ms, service: com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI, method: batchSaveEmployee
    at org.apache.dubbo.rpc.TriRpcStatus.asException(TriRpcStatus.java:214)
    at org.apache.dubbo.rpc.protocol.tri.DeadlineFuture$TimeoutCheckTask.notifyTimeout(DeadlineFuture.java:183)
    at org.apache.dubbo.rpc.protocol.tri.DeadlineFuture$TimeoutCheckTask.lambda$run$0(DeadlineFuture.java:169)
    at org.apache.dubbo.common.threadpool.ThreadlessExecutor$RunnableWrapper.run(ThreadlessExecutor.java:184)
    at org.apache.dubbo.common.threadpool.ThreadlessExecutor.waitAndDrain(ThreadlessExecutor.java:103)
    at org.apache.dubbo.rpc.AsyncRpcResult.get(AsyncRpcResult.java:194)
    at org.apache.dubbo.rpc.protocol.AbstractInvoker.waitForResultIfSync(AbstractInvoker.java:266)
    at org.apache.dubbo.rpc.protocol.AbstractInvoker.invoke(AbstractInvoker.java:186)
    at org.apache.dubbo.rpc.listener.ListenerInvokerWrapper.invoke(ListenerInvokerWrapper.java:71)
    at com.emax.zhenghe.common.filters.ParameterLogFilter.invoke(ParameterLogFilter.java:28)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:327)
    at com.serviceshare.dubbo.auth.DubboConsumerAuthFilter.invoke(DubboConsumerAuthFilter.java:48)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:327)
    at com.serviceshare.principal.dubbo.filter.ClientPrincipalFilter.invoke(ClientPrincipalFilter.java:33)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:327)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CallbackRegistrationInvoker.invoke(FilterChainBuilder.java:194)
    at org.apache.dubbo.rpc.protocol.ReferenceCountInvokerWrapper.invoke(ReferenceCountInvokerWrapper.java:78)
    at org.apache.dubbo.rpc.cluster.support.AbstractClusterInvoker.invokeWithContext(AbstractClusterInvoker.java:379)
    at org.apache.dubbo.rpc.cluster.support.FailoverClusterInvoker.doInvoke(FailoverClusterInvoker.java:81)
    at org.apache.dubbo.rpc.cluster.support.AbstractClusterInvoker.invoke(AbstractClusterInvoker.java:341)
    at org.apache.dubbo.rpc.cluster.router.RouterSnapshotFilter.invoke(RouterSnapshotFilter.java:46)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:327)
    at org.apache.dubbo.monitor.support.MonitorFilter.invoke(MonitorFilter.java:100)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:327)
    at org.apache.dubbo.rpc.protocol.dubbo.filter.FutureFilter.invoke(FutureFilter.java:52)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:327)
    at org.apache.dubbo.rpc.cluster.filter.support.ConsumerClassLoaderFilter.invoke(ConsumerClassLoaderFilter.java:40)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:327)
    at org.apache.dubbo.rpc.cluster.filter.support.ConsumerContextFilter.invoke(ConsumerContextFilter.java:120)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:327)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CallbackRegistrationInvoker.invoke(FilterChainBuilder.java:194)
    at org.apache.dubbo.rpc.cluster.support.wrapper.AbstractCluster$ClusterFilterInvoker.invoke(AbstractCluster.java:92)
    at org.apache.dubbo.rpc.cluster.support.wrapper.MockClusterInvoker.invoke(MockClusterInvoker.java:103)
    at org.apache.dubbo.registry.client.migration.MigrationInvoker.invoke(MigrationInvoker.java:284)
    at org.apache.dubbo.rpc.proxy.InvocationUtil.invoke(InvocationUtil.java:57)
    at org.apache.dubbo.rpc.proxy.InvokerInvocationHandler.invoke(InvokerInvocationHandler.java:75)
    at com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPIDubboProxy22.batchSaveEmployee(CloudCaringEmployeeAPIDubboProxy22.java)
    at sun.reflect.GeneratedMethodAccessor403.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:205)
    at com.sun.proxy.$Proxy184.batchSaveEmployee(Unknown Source)
    at com.sp.oms.webapi.modules.cloud.caring.EmployeeController.batchSave(EmployeeController.java:231)
    at sun.reflect.GeneratedMethodAccessor402.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:92)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:93)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1590)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:750)

Server-side evidence


2024-01-25 14:49:10.571 [TID: N/A] [] [DubboServerHandler-192.168.48.17:20880-thread-10] DEBUG org.apache.dubbo.remoting.transport.DecodeHandler:? -  [DUBBO] Decode decodeable message org.apache.dubbo.rpc.protocol.dubbo.DecodeableRpcInvocation, dubbo version: 3.1.11, current host: 192.168.48.17
2024-01-25 14:49:10.571 [TID: N/A] [] [DubboServerHandler-192.168.48.17:20880-thread-10] INFO  c.e.zhenghe.common.filters.ProviderRecordLogFilter:? - Receive request: [dubbo:192.168.48.17:20880:cloud-caring:instance:test]: from sp-oms-consumer (unknown/unknown:64395) to com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI(batchSaveEmployee), args [1706164860382250]
2024-01-25 14:49:10.571 [TID: N/A] [] [DubboServerHandler-192.168.48.17:20880-thread-10] INFO  c.e.zhenghe.common.filters.ProviderRecordLogFilter:? - Process request complete: com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI(batchSaveEmployee), return: null, elapsed: 0ms
2024-01-25 14:49:10.572 [TID: N/A] [] [DubboServerHandler-192.168.48.17:20880-thread-10] ERROR org.apache.dubbo.rpc.filter.ExceptionFilter:? -  [DUBBO] Got unchecked and undeclared exception which called by 192.168.48.17. service: com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI, method: batchSaveEmployee, exception: org.springframework.data.redis.serializer.SerializationException: fileUploadKey is null, dubbo version: 3.1.11, current host: 192.168.48.17, error code: 5-36. This may be caused by , go to https://dubbo.apache.org/faq/5/36 to find instructions. org.springframework.data.redis.serializer.SerializationException: fileUploadKey is null

...

2024-01-25 14:49:37.410 [TID: N/A] [] [tri-protocol-50051-thread-10] INFO  c.e.zhenghe.common.filters.ProviderRecordLogFilter:? - Receive request: [tri:192.168.48.17:50051:cloud-caring:instance:test]: from sp-oms-consumer (unknown/unknown:63939) to com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI(batchSaveEmployee), args [1706164860382250]
2024-01-25 14:49:37.410 [TID: N/A] [] [tri-protocol-50051-thread-10] INFO  c.e.zhenghe.common.filters.ProviderRecordLogFilter:? - Process request complete: com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI(batchSaveEmployee), return: null, elapsed: 0ms
2024-01-25 14:49:37.410 [TID: N/A] [] [tri-protocol-50051-thread-10] ERROR org.apache.dubbo.rpc.filter.ExceptionFilter:? -  [DUBBO] Got unchecked and undeclared exception which called by 192.168.48.17. service: com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI, method: batchSaveEmployee, exception: org.springframework.data.redis.serializer.SerializationException: fileUploadKey is null, dubbo version: 3.1.11, current host: 192.168.48.17, error code: 5-36. This may be caused by , go to https://dubbo.apache.org/faq/5/36 to find instructions. org.springframework.data.redis.serializer.SerializationException: fileUploadKey is null
iwangjie commented 10 months ago

不太了解源码，只能提供一个信息，正常调用不会走：org.apache.dubbo.rpc.protocol.tri.call.AbstractServerCallListener#invoke 逻辑，异常调用会走到 ：org.apache.dubbo.rpc.protocol.tri.call.AbstractServerCallListener#invoke

iwangjie commented 10 months ago

不好意思，补充信息，似乎 tri 的会有问题，Dubbo 的不会有问题，是不是因为协议复用呢？

2024-01-25 15:11:39.337 [TID: N/A] [] [DubboServerHandler-192.168.48.17:20880-thread-31] DEBUG org.apache.dubbo.remoting.transport.DecodeHandler:? -  [DUBBO] Decode decodeable message org.apache.dubbo.rpc.protocol.dubbo.DecodeableRpcInvocation, dubbo version: 3.1.11, current host: 192.168.48.17
2024-01-25 15:11:39.337 [TID: N/A] [] [DubboServerHandler-192.168.48.17:20880-thread-31] INFO  c.e.zhenghe.common.filters.ProviderRecordLogFilter:? - Receive request: [dubbo:192.168.48.17:20880:cloud-caring:instance:test]: from sp-oms-consumer (unknown/unknown:54736) to com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI(batchSaveEmployee), args [1706164860382250]
2024-01-25 15:11:39.337 [TID: N/A] [] [DubboServerHandler-192.168.48.17:20880-thread-31] INFO  c.e.zhenghe.common.filters.ProviderRecordLogFilter:? - Process request complete: com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI(batchSaveEmployee), return: null, elapsed: 0ms
2024-01-25 15:11:39.337 [TID: N/A] [] [DubboServerHandler-192.168.48.17:20880-thread-31] ERROR org.apache.dubbo.rpc.filter.ExceptionFilter:? -  [DUBBO] Got unchecked and undeclared exception which called by 192.168.48.17. service: com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI, method: batchSaveEmployee, exception: org.springframework.data.redis.serializer.SerializationException: fileUploadKey is null, dubbo version: 3.1.11, current host: 192.168.48.17, error code: 5-36. This may be caused by , go to https://dubbo.apache.org/faq/5/36 to find instructions. org.springframework.data.redis.serializer.SerializationException: fileUploadKey is null

2024-01-25 15:11:42.138 [TID: N/A] [] [DubboServerHandler-192.168.48.17:20880-thread-32] DEBUG org.apache.dubbo.remoting.transport.DecodeHandler:? -  [DUBBO] Decode decodeable message org.apache.dubbo.rpc.protocol.dubbo.DecodeableRpcInvocation, dubbo version: 3.1.11, current host: 192.168.48.17
2024-01-25 15:11:42.138 [TID: N/A] [] [DubboServerHandler-192.168.48.17:20880-thread-32] INFO  c.e.zhenghe.common.filters.ProviderRecordLogFilter:? - Receive request: [dubbo:192.168.48.17:20880:cloud-caring:instance:test]: from sp-oms-consumer (unknown/unknown:54736) to com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI(batchSaveEmployee), args [1706164860382250]
2024-01-25 15:11:42.138 [TID: N/A] [] [DubboServerHandler-192.168.48.17:20880-thread-32] INFO  c.e.zhenghe.common.filters.ProviderRecordLogFilter:? - Process request complete: com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI(batchSaveEmployee), return: null, elapsed: 0ms
2024-01-25 15:11:42.138 [TID: N/A] [] [DubboServerHandler-192.168.48.17:20880-thread-32] ERROR org.apache.dubbo.rpc.filter.ExceptionFilter:? -  [DUBBO] Got unchecked and undeclared exception which called by 192.168.48.17. service: com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI, method: batchSaveEmployee, exception: org.springframework.data.redis.serializer.SerializationException: fileUploadKey is null, dubbo version: 3.1.11, current host: 192.168.48.17, error code: 5-36. This may be caused by , go to https://dubbo.apache.org/faq/5/36 to find instructions. org.springframework.data.redis.serializer.SerializationException: fileUploadKey is null

2024-01-25 15:11:43.226 [TID: N/A] [] [DubboServerHandler-192.168.48.17:20880-thread-33] DEBUG org.apache.dubbo.remoting.transport.DecodeHandler:? -  [DUBBO] Decode decodeable message org.apache.dubbo.rpc.protocol.dubbo.DecodeableRpcInvocation, dubbo version: 3.1.11, current host: 192.168.48.17
2024-01-25 15:11:43.226 [TID: N/A] [] [DubboServerHandler-192.168.48.17:20880-thread-33] INFO  c.e.zhenghe.common.filters.ProviderRecordLogFilter:? - Receive request: [dubbo:192.168.48.17:20880:cloud-caring:instance:test]: from sp-oms-consumer (unknown/unknown:54736) to com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI(batchSaveEmployee), args [1706164860382250]
2024-01-25 15:11:43.226 [TID: N/A] [] [DubboServerHandler-192.168.48.17:20880-thread-33] INFO  c.e.zhenghe.common.filters.ProviderRecordLogFilter:? - Process request complete: com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI(batchSaveEmployee), return: null, elapsed: 0ms
2024-01-25 15:11:43.226 [TID: N/A] [] [DubboServerHandler-192.168.48.17:20880-thread-33] ERROR org.apache.dubbo.rpc.filter.ExceptionFilter:? -  [DUBBO] Got unchecked and undeclared exception which called by 192.168.48.17. service: com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI, method: batchSaveEmployee, exception: org.springframework.data.redis.serializer.SerializationException: fileUploadKey is null, dubbo version: 3.1.11, current host: 192.168.48.17, error code: 5-36. This may be caused by , go to https://dubbo.apache.org/faq/5/36 to find instructions. org.springframework.data.redis.serializer.SerializationException: fileUploadKey is null

2024-01-25 15:11:47.808 [TID: N/A] [] [tri-protocol-50051-thread-24] INFO  c.e.zhenghe.common.filters.ProviderRecordLogFilter:? - Receive request: [tri:192.168.48.17:50051:cloud-caring:instance:test]: from sp-oms-consumer (unknown/unknown:54481) to com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI(batchSaveEmployee), args [1706164860382250]
2024-01-25 15:11:47.808 [TID: N/A] [] [tri-protocol-50051-thread-24] INFO  c.e.zhenghe.common.filters.ProviderRecordLogFilter:? - Process request complete: com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI(batchSaveEmployee), return: null, elapsed: 0ms
2024-01-25 15:11:47.808 [TID: N/A] [] [tri-protocol-50051-thread-24] ERROR org.apache.dubbo.rpc.filter.ExceptionFilter:? -  [DUBBO] Got unchecked and undeclared exception which called by 192.168.48.17. service: com.serviceshare.cloud.caring.rpcapi.CloudCaringEmployeeAPI, method: batchSaveEmployee, exception: org.springframework.data.redis.serializer.SerializationException: fileUploadKey is null, dubbo version: 3.1.11, current host: 192.168.48.17, error code: 5-36. This may be caused by , go to https://dubbo.apache.org/faq/5/36 to find instructions. org.springframework.data.redis.serializer.SerializationException: fileUploadKey is null
AlbumenJ commented 9 months ago

Can you please submit a demo?

iwangjie commented 9 months ago

Can you please submit a demo?

已经解决，原因： org.apache.dubbo.rpc.protocol.tri.stream.TripleServerStream#complete 抛出异常 Method threw 'java.lang.NoSuchMethodError' exception. com.google.protobuf.GeneratedMessageV3.isStringEmpty(Ljava/lang/Object;)Z

因为 mysql8 驱动引入了 protobuf 依赖，和 Dubbo 这块代码冲突，没有使用 protobuf 需求，规约 protobuf 版本即可。但是 Dubbo直接吞掉异常不响应如果可以，最好优化一下

iwangjie commented 9 months ago

Can you please submit a demo?

另外，建议 Dubbo 3 传递 protobuf 依赖，或者文档注明应该使用的兼容版本是多少🥲。