sermant-io / Sermant

A Cloud-Native Proxyless Service Mesh based on Java Bytecode Enhancement Technology
https://sermant.io/
Apache License 2.0
1.26k stars 170 forks source link

DubboProviderInterceptor will remove traffic tag after invocation on consumer side, it will cause the follow up invocation can not get traffic tag from ThreadLocal when the traffic tag used for traffic route later #1654

Closed rztao closed 3 weeks ago

rztao commented 1 month ago

What happened?

DubboProviderInterceptor will be invoked in consumer side(MonitorFilter is enable in both provider and consumer side in default), it will cause previous traffic tag removed and the follow up invocation can not get traffic tag from ThreadLocal when the traffic tag used for traffic route later if remove traffic tag unconditionally.

Here is stack for real case and the route later which use traffic tag does not work because the traffic tag is removed by DubboProviderInterceptor method previously executed.

ts=2024-10-22 17:20:38.674;thread_name=rpc-core-13;id=754;is_daemon=false;priority=5;TCCL=org.springframework.boot.web.embedded.tomcat.TomcatEmbeddedWebappClassLoader@3a1eb893 @com.huaweicloud.sermant.core.utils.tag.TrafficUtils.removeTrafficTag() at com.huaweicloud.sermant.tag.transmission.dubbov3.interceptors.ApacheDubboProviderInterceptor.doAfter(ApacheDubboProviderInterceptor.java:86) at com.huaweicloud.sermant.tag.transmission.interceptors.AbstractServerInterceptor.after(AbstractServerInterceptor.java:58) at com.huaweicloud.sermant.core.plugin.agent.template.BaseAdviseHandler.handleMethodExit(BaseAdviseHandler.java:169) at com.huaweicloud.sermant.core.plugin.agent.template.BaseAdviseHandler.handleMethodExit(BaseAdviseHandler.java:130) at com.huaweicloud.sermant.core.plugin.agent.template.DefaultAdviser.onMethodExit(DefaultAdviser.java:63) at com.huaweicloud.sermant.core.plugin.agent.adviser.AdviserScheduler.onMethodExit(AdviserScheduler.java:96) at org.apache.dubbo.monitor.support.MonitorFilter.invoke(MonitorFilter.java:91) at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61) at org.apache.dubbo.rpc.protocol.dubbo.filter.FutureFilter.invoke(FutureFilter.java:52) at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61) at org.apache.dubbo.rpc.filter.ConsumerContextFilter.invoke(ConsumerContextFilter.java:69) at org.apache.dubbo.rpc.protocol.FilterNode.invoke(FilterNode.java:61) at org.apache.dubbo.rpc.protocol.InvokerWrapper.invoke(InvokerWrapper.java:56) at org.apache.dubbo.rpc.cluster.support.FailoverClusterInvoker.doInvoke(FailoverClusterInvoker.java:82) at org.apache.dubbo.rpc.cluster.support.AbstractClusterInvoker.invoke(AbstractClusterInvoker.java:265) at org.apache.dubbo.rpc.cluster.interceptor.ClusterInterceptor.intercept(ClusterInterceptor.java:47) at org.apache.dubbo.rpc.cluster.support.wrapper.AbstractCluster$InterceptorInvokerNode.invoke(AbstractCluster.java:92) at org.apache.dubbo.rpc.cluster.support.wrapper.MockClusterInvoker.invoke(MockClusterInvoker.java:93) at org.apache.dubbo.registry.client.migration.MigrationInvoker.invoke(MigrationInvoker.java:169) at org.apache.dubbo.rpc.cluster.support.registry.ZoneAwareClusterInvoker.doInvoke(ZoneAwareClusterInvoker.java:105) at org.apache.dubbo.rpc.cluster.support.AbstractClusterInvoker.invoke(AbstractClusterInvoker.java:265) at org.apache.dubbo.rpc.cluster.interceptor.ClusterInterceptor.intercept(ClusterInterceptor.java:47) at org.apache.dubbo.rpc.cluster.support.wrapper.AbstractCluster$InterceptorInvokerNode.invoke(AbstractCluster.java:92) at org.apache.dubbo.rpc.proxy.InvokerInvocationHandler.invoke(InvokerInvocationHandler.java:96) at org.apache.dubbo.common.bytecode.proxy4.queryPage(proxy4.java:-1) ..

How can we reproduce it (as minimally and precisely as possible)?

Do multiple dubbo invocations in on request, and get traffic tag between them, it will get no traffic tags sent with the request.

Anything else we need to know?

No response

Sermant version

<= v2.1.0

OS version

No response

hanbingleixue commented 3 weeks ago

The fixed code has been incorporated and will be released in later versions. Therefore, this issue is closed.