search

apache / dubbo

The java implementation of Apache Dubbo. An RPC and microservice framework.
https://dubbo.apache.org/
Apache License 2.0
40.51k stars 26.43k forks source link

系统启动后的一段时间内，provider错乱 #10717

Closed cwt9562 closed 2 years ago

cwt9562 commented 2 years ago

Environment

Steps to reproduce this issue

  1. 使用zk作为注册中心
  2. 使用org.apache.dubbo:dubbo-spring-boot-starter
  3. xxx

Pls. provide [GitHub address] to reproduce this issue.

Expected Behavior

Actual Behavior

系统启动的15s内，provider错乱 具体表现可见下方日志 调用com.alibaba.rdc.basic.system.dict.DictProvider的findAll方法 报错却说找不到UserProvider的findAll

If there is an exception, please attach the exception trace:

2022-10-09 17:22:08.162 ERROR 20140 --- [  XNIO-2 task-1] c.l.rdc.core.dubbo.DubboExceptionFilter  :  [DUBBO] Got unchecked and undeclared exception which called by null. service: com.alibaba.rdc.basic.system.dict.DictProvider, method: findAll, exception: org.apache.dubbo.rpc.RpcException: Failed to invoke remote method: findAll, provider: DefaultServiceInstance{serviceName='demo-basic', host='192.168.1.100', port=20881, enabled=true, healthy=true, metadata={dubbo.endpoints=[{"port":20881,"protocol":"dubbo"}], dubbo.metadata-service.url-params={"connections":"1","timeout":"5000","version":"1.0.0","dubbo":"2.0.2","release":"3.0.12","side":"provider","port":"20881","protocol":"dubbo"}, dubbo.metadata.revision=86167ff15d98ad75c64ec137880be22b, dubbo.metadata.storage-type=local}}, cause: org.apache.dubbo.remoting.RemotingException: org.apache.dubbo.rpc.RpcException: Failed to invoke remote proxy method findAll to registry://localhost:2181/org.apache.dubbo.registry.RegistryService?application=demo-basic&dubbo=2.0.2&group=demo&logger=slf4j&pid=3752&qos.enable=false&registry=zookeeper&release=3.0.12&simplified=true&timestamp=1665307305713, cause: Not found method "findAll" in class com.alibaba.rdc.basic.system.user.UserProvider.
org.apache.dubbo.rpc.RpcException: Failed to invoke remote proxy method findAll to registry://localhost:2181/org.apache.dubbo.registry.RegistryService?application=demo-basic&dubbo=2.0.2&group=demo&logger=slf4j&pid=3752&qos.enable=false&registry=zookeeper&release=3.0.12&simplified=true&timestamp=1665307305713, cause: Not found method "findAll" in class com.alibaba.rdc.basic.system.user.UserProvider.
    at org.apache.dubbo.rpc.proxy.AbstractProxyInvoker.invoke(AbstractProxyInvoker.java:134)
    at org.apache.dubbo.config.invoker.DelegateProviderMetaDataInvoker.invoke(DelegateProviderMetaDataInvoker.java:55)
    at org.apache.dubbo.rpc.protocol.InvokerWrapper.invoke(InvokerWrapper.java:56)
    at org.apache.dubbo.rpc.filter.ClassLoaderCallbackFilter.invoke(ClassLoaderCallbackFilter.java:38)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at org.apache.dubbo.rpc.protocol.dubbo.filter.TraceFilter.invoke(TraceFilter.java:77)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at org.apache.dubbo.rpc.filter.TimeoutFilter.invoke(TimeoutFilter.java:44)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at org.apache.dubbo.monitor.support.MonitorFilter.invoke(MonitorFilter.java:99)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at com.alibaba.rdc.core.dubbo.DubboExceptionFilter.invoke(DubboExceptionFilter.java:26)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at org.apache.dubbo.rpc.filter.GenericFilter.invoke(GenericFilter.java:191)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at org.apache.dubbo.rpc.filter.ClassLoaderFilter.invoke(ClassLoaderFilter.java:54)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at org.apache.dubbo.rpc.filter.EchoFilter.invoke(EchoFilter.java:41)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at org.apache.dubbo.rpc.filter.ProfilerServerFilter.invoke(ProfilerServerFilter.java:56)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at org.apache.dubbo.rpc.filter.ContextFilter.invoke(ContextFilter.java:131)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CallbackRegistrationInvoker.invoke(FilterChainBuilder.java:193)
    at org.apache.dubbo.rpc.protocol.dubbo.DubboProtocol$1.reply(DubboProtocol.java:151)
    at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.handleRequest(HeaderExchangeHandler.java:100)
    at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.received(HeaderExchangeHandler.java:175)
    at org.apache.dubbo.remoting.transport.DecodeHandler.received(DecodeHandler.java:51)
    at org.apache.dubbo.remoting.transport.dispatcher.ChannelEventRunnable.run(ChannelEventRunnable.java:59)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at org.apache.dubbo.common.threadlocal.InternalRunnable.run(InternalRunnable.java:41)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.apache.dubbo.common.bytecode.NoSuchMethodException: Not found method "findAll" in class com.alibaba.rdc.basic.system.user.UserProvider.
    at com.alibaba.rdc.basic.system.user.UserProviderDubboWrap3.invokeMethod(UserProviderDubboWrap3.java)
    at org.apache.dubbo.rpc.proxy.javassist.JavassistProxyFactory$1.doInvoke(JavassistProxyFactory.java:71)
    at org.apache.dubbo.rpc.proxy.AbstractProxyInvoker.invoke(AbstractProxyInvoker.java:99)
    ... 32 more
, dubbo version: 3.0.12, current host: 192.168.1.100
Just put your stack trace here!
AlbumenJ commented 2 years ago

是不是有部分机器发布的 com.alibaba.rdc.basic.system.user.UserProvider 接口不存在 findAll 方法。 Dubbo 做服务发现的时候是以接口为粒度的，如果添加接口后没有全部机器都部署就调用时会导致找不到方法的问题，因为 Dubbo 作为消费端的时候是不知道服务端具体真的发了多少个方法的，只知道发了某个服务。

cwt9562 commented 2 years ago

并没有，我是在我本机上的开发环境，就一个provider 一个consumer provider侧启动完毕的，再启动的consumer侧 出问题的不仅仅DictProvider的findAll方法 其他provider的其他方法（比如 RoleProvider的list方法），在启动后的一段时间内（体感30s），也会报错说 找不到UserProvider的list方法

cwt9562 commented 2 years ago

这是另一个报错堆栈 可以看到 这次是AuthLogProvider.logSuccessfulLogin() 报错说找不到UserProvider的logSuccessfulLogin

2022-10-10 10:53:04.580 ERROR 12392 --- [        async-1] c.l.rdc.core.dubbo.DubboExceptionFilter  :  [DUBBO] Got unchecked and undeclared exception which called by null. service: com.alibaba.rdc.basic.system.authlog.AuthLogProvider, method: logSuccessfulLogin, exception: org.apache.dubbo.rpc.RpcException: Failed to invoke remote method: logSuccessfulLogin, provider: DefaultServiceInstance{serviceName='project-template-basic', host='192.168.1.111', port=20881, enabled=true, healthy=true, metadata={dubbo.endpoints=[{"port":20881,"protocol":"dubbo"}], dubbo.metadata-service.url-params={"connections":"1","timeout":"5000","version":"1.0.0","dubbo":"2.0.2","release":"3.0.12","side":"provider","port":"20881","protocol":"dubbo"}, dubbo.metadata.revision=86167ff15d98ad75c64ec137880be22b, dubbo.metadata.storage-type=local}}, cause: org.apache.dubbo.remoting.RemotingException: org.apache.dubbo.rpc.RpcException: Failed to invoke remote proxy method logSuccessfulLogin to registry://localhost:2181/org.apache.dubbo.registry.RegistryService?application=project-template-basic&dubbo=2.0.2&group=project-template&logger=slf4j&pid=11320&qos.enable=false&registry=zookeeper&release=3.0.12&simplified=true&timestamp=1665370348081, cause: Not found method "logSuccessfulLogin" in class com.alibaba.rdc.basic.system.user.UserProvider.
org.apache.dubbo.rpc.RpcException: Failed to invoke remote proxy method logSuccessfulLogin to registry://localhost:2181/org.apache.dubbo.registry.RegistryService?application=project-template-basic&dubbo=2.0.2&group=project-template&logger=slf4j&pid=11320&qos.enable=false&registry=zookeeper&release=3.0.12&simplified=true&timestamp=1665370348081, cause: Not found method "logSuccessfulLogin" in class com.alibaba.rdc.basic.system.user.UserProvider.
    at org.apache.dubbo.rpc.proxy.AbstractProxyInvoker.invoke(AbstractProxyInvoker.java:134)
    at org.apache.dubbo.config.invoker.DelegateProviderMetaDataInvoker.invoke(DelegateProviderMetaDataInvoker.java:55)
    at org.apache.dubbo.rpc.protocol.InvokerWrapper.invoke(InvokerWrapper.java:56)
    at org.apache.dubbo.rpc.filter.ClassLoaderCallbackFilter.invoke(ClassLoaderCallbackFilter.java:38)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at org.apache.dubbo.rpc.protocol.dubbo.filter.TraceFilter.invoke(TraceFilter.java:77)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at org.apache.dubbo.rpc.filter.TimeoutFilter.invoke(TimeoutFilter.java:44)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at org.apache.dubbo.monitor.support.MonitorFilter.invoke(MonitorFilter.java:99)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at com.alibaba.rdc.core.dubbo.DubboExceptionFilter.invoke(DubboExceptionFilter.java:26)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at org.apache.dubbo.rpc.filter.GenericFilter.invoke(GenericFilter.java:191)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at org.apache.dubbo.rpc.filter.ClassLoaderFilter.invoke(ClassLoaderFilter.java:54)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at org.apache.dubbo.rpc.filter.EchoFilter.invoke(EchoFilter.java:41)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at org.apache.dubbo.rpc.filter.ProfilerServerFilter.invoke(ProfilerServerFilter.java:56)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at org.apache.dubbo.rpc.filter.ContextFilter.invoke(ContextFilter.java:131)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:326)
    at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CallbackRegistrationInvoker.invoke(FilterChainBuilder.java:193)
    at org.apache.dubbo.rpc.protocol.dubbo.DubboProtocol$1.reply(DubboProtocol.java:151)
    at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.handleRequest(HeaderExchangeHandler.java:100)
    at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.received(HeaderExchangeHandler.java:175)
    at org.apache.dubbo.remoting.transport.DecodeHandler.received(DecodeHandler.java:51)
    at org.apache.dubbo.remoting.transport.dispatcher.ChannelEventRunnable.run(ChannelEventRunnable.java:59)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at org.apache.dubbo.common.threadlocal.InternalRunnable.run(InternalRunnable.java:41)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.apache.dubbo.common.bytecode.NoSuchMethodException: Not found method "logSuccessfulLogin" in class com.alibaba.rdc.basic.system.user.UserProvider.
    at com.alibaba.rdc.basic.system.user.UserProviderDubboWrap3.invokeMethod(UserProviderDubboWrap3.java)
    at org.apache.dubbo.rpc.proxy.javassist.JavassistProxyFactory$1.doInvoke(JavassistProxyFactory.java:71)
    at org.apache.dubbo.rpc.proxy.AbstractProxyInvoker.invoke(AbstractProxyInvoker.java:99)
    ... 32 more
, dubbo version: 3.0.12, current host: 192.168.1.111
AlbumenJ commented 2 years ago

arthas 上去 10.230.250.210 这台机器看下这个方法是否存在。

对应的日志：serviceName='project-template-basic', host='10.230.250.210',

cwt9562 commented 2 years ago

我的consumer侧还有一些警告

2022-10-10 10:52:21.571  WARN 12392 --- [           main] o.a.d.r.integration.RegistryDirectory    :  [DUBBO] Servicecom.alibaba.rdc.basic.system.rbac.roleuser.RoleUserMappingProvider received empty address list with no EMPTY protocol set, trigger empty protection., dubbo version: 3.0.12, current host: 192.168.1.111
2022-10-10 10:52:21.767  WARN 12392 --- [           main] .d.r.c.ServiceDiscoveryRegistryDirectory :  [DUBBO] Received empty url list, will ignore for protection purpose., dubbo version: 3.0.12, current host: 192.168.1.111
2022-10-10 10:52:21.806  WARN 12392 --- [           main] o.a.d.r.integration.RegistryDirectory    :  [DUBBO] Servicecom.alibaba.rdc.basic.system.job.ScheduleJobProvider received empty address list with no EMPTY protocol set, trigger empty protection., dubbo version: 3.0.12, current host: 192.168.1.111
2022-10-10 10:52:21.837  WARN 12392 --- [           main] o.a.d.r.integration.RegistryDirectory    :  [DUBBO] Servicecom.alibaba.rdc.basic.system.user.UserProvider received empty address list with no EMPTY protocol set, trigger empty protection., dubbo version: 3.0.12, current host: 192.168.1.111
2022-10-10 10:52:21.866  WARN 12392 --- [           main] o.a.d.r.integration.RegistryDirectory    :  [DUBBO] Servicecom.alibaba.rdc.basic.system.optlog.OptLogProvider received empty address list with no EMPTY protocol set, trigger empty protection., dubbo version: 3.0.12, current host: 192.168.1.111
2022-10-10 10:52:21.896  WARN 12392 --- [           main] o.a.d.r.integration.RegistryDirectory    :  [DUBBO] Servicecom.alibaba.rdc.basic.system.authlog.AuthLogProvider received empty address list with no EMPTY protocol set, trigger empty protection., dubbo version: 3.0.12, current host: 192.168.1.111
2022-10-10 10:52:21.916  WARN 12392 --- [           main] o.a.d.r.integration.RegistryDirectory    :  [DUBBO] Servicecom.alibaba.rdc.basic.system.rbac.permission.PermissionProvider received empty address list with no EMPTY protocol set, trigger empty protection., dubbo version: 3.0.12, current host: 192.168.1.111
2022-10-10 10:52:21.947  WARN 12392 --- [           main] o.a.d.r.integration.RegistryDirectory    :  [DUBBO] Servicecom.alibaba.rdc.basic.system.rbac.rolepermission.RolePermissionMappingProvider received empty address list with no EMPTY protocol set, trigger empty protection., dubbo version: 3.0.12, current host: 192.168.1.111
2022-10-10 10:52:21.966  WARN 12392 --- [           main] o.a.d.r.integration.RegistryDirectory    :  [DUBBO] Servicecom.alibaba.rdc.basic.idaas.user.UserIdaasProvider received empty address list with no EMPTY protocol set, trigger empty protection., dubbo version: 3.0.12, current host: 192.168.1.111
2022-10-10 10:52:21.987  WARN 12392 --- [           main] o.a.d.r.integration.RegistryDirectory    :  [DUBBO] Servicecom.alibaba.rdc.basic.system.prop.SystemPropertyProvider received empty address list with no EMPTY protocol set, trigger empty protection., dubbo version: 3.0.12, current host: 192.168.1.111
2022-10-10 10:52:22.016  WARN 12392 --- [           main] o.a.d.r.integration.RegistryDirectory    :  [DUBBO] Servicecom.alibaba.rdc.basic.idaas.org.OrgIdaasProvider received empty address list with no EMPTY protocol set, trigger empty protection., dubbo version: 3.0.12, current host: 192.168.1.111
2022-10-10 10:52:22.043  WARN 12392 --- [           main] o.a.d.r.integration.RegistryDirectory    :  [DUBBO] Servicecom.alibaba.rdc.basic.system.message.MessageLogProvider received empty address list with no EMPTY protocol set, trigger empty protection., dubbo version: 3.0.12, current host: 192.168.1.111
2022-10-10 10:52:22.066  WARN 12392 --- [           main] o.a.d.r.integration.RegistryDirectory    :  [DUBBO] Servicecom.alibaba.rdc.basic.system.org.OrgProvider received empty address list with no EMPTY protocol set, trigger empty protection., dubbo version: 3.0.12, current host: 192.168.1.111
2022-10-10 10:52:22.093  WARN 12392 --- [           main] o.a.d.r.integration.RegistryDirectory    :  [DUBBO] Servicecom.alibaba.rdc.basic.system.optlog.OptLogDetailProvider received empty address list with no EMPTY protocol set, trigger empty protection., dubbo version: 3.0.12, current host: 192.168.1.111
2022-10-10 10:52:22.117  WARN 12392 --- [           main] o.a.d.r.integration.RegistryDirectory    :  [DUBBO] Servicecom.alibaba.rdc.basic.system.rbac.role.RoleProvider received empty address list with no EMPTY protocol set, trigger empty protection., dubbo version: 3.0.12, current host: 192.168.1.111
2022-10-10 10:52:22.137  WARN 12392 --- [           main] o.a.d.r.integration.RegistryDirectory    :  [DUBBO] Servicecom.alibaba.rdc.basic.demo.DemoProvider received empty address list with no EMPTY protocol set, trigger empty protection., dubbo version: 3.0.12, current host: 192.168.1.111
2022-10-10 10:52:22.156  WARN 12392 --- [           main] o.a.d.r.integration.RegistryDirectory    :  [DUBBO] Servicecom.alibaba.rdc.basic.system.dict.DictItemProvider received empty address list with no EMPTY protocol set, trigger empty protection., dubbo version: 3.0.12, current host: 192.168.1.111
2022-10-10 10:52:22.176  WARN 12392 --- [           main] o.a.d.r.integration.RegistryDirectory    :  [DUBBO] Servicecom.alibaba.rdc.basic.system.dict.DictProvider received empty address list with no EMPTY protocol set, trigger empty protection., dubbo version: 3.0.12, current host: 192.168.1.111
2022-10-10 10:52:24.308  WARN 12392 --- [           main] o.a.d.r.client.AbstractServiceDiscovery  :  [DUBBO] No valid instance found, stop registering instance address to registry., dubbo version: 3.0.12, current host: 192.168.1.111
cwt9562 commented 2 years ago

arthas 上去 10.230.250.210 这台机器看下这个方法是否存在。

对应的日志：serviceName='project-template-basic', host='10.230.250.210',

我不太理解，你想看哪个方法？ 我的UserProvider（这是一个api定义接口）当然没有findAll()和logSuccessfulLogin() 因为findAll()在DictProvider，logSuccessfulLogin()在AuthLogProvider

cwt9562 commented 2 years ago

补充： 我使用的是spring-cloud-starter-zookeeper-discovery + dubbo-spring-boot-starter

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-zookeeper-discovery</artifactId>
        </dependency>

        <dependency>
            <groupId>org.apache.dubbo</groupId>
            <artifactId>dubbo-spring-boot-starter</artifactId>
        </dependency>

springboot版本：2.7.4 springcloud版本：2021.0.4 spring-cloud-zookeeper版本：3.1.3 dubbo版本：3.0.12

dubbo配置文件用的zk

zookeeper:
  address: localhost:2181

spring:
  cloud:
    zookeeper:
      connect-string: ${zookeeper.address}

dubbo:
  registry:
    address: "zookeeper://${zookeeper.address}"
AlbumenJ commented 2 years ago

因为findAll()在DictProvider，logSuccessfulLogin()在AuthLogProvider

然后对应的 Java 接口一样吗（UserProvider）

AlbumenJ commented 2 years ago

如果对应的 Interface 一样是不允许的，从 Dubbo 的角度只能识别接口，接口内部无法再拆了。这种场景是需要拆分成多个 Interface 的

lixiaojiee commented 2 years ago

@cwt9562 可否将完整的示例工程提交到您的个人仓库，并在此加以引用？

cwt9562 commented 2 years ago

如果对应的 Interface 一样是不允许的，从 Dubbo 的角度只能识别接口，接口内部无法再拆了。这种场景是需要拆分成多个 Interface 的

这三个provider类，都继承自一个基础的interface 像这样

public interface UserProvider extends BaseCrudlProvider<User> {

}

public interface DictProvider extends BaseCrudlProvider<Dict> {
    List<B> findAll();
}

public interface BaseCrudlProvider<B> {

    String create(B bean);

    void delete(String id);

    void update(B bean);

    B get(String id);

    List<B> list(List<String> ids);

}
cwt9562 commented 2 years ago

为啥你们要关闭这个issue，明明问题还没有找到

lixiaojiee commented 2 years ago

为啥你们要关闭这个issue，明明问题还没有找到

应该是操作错了，我也正在尝试复现这个问题

liufeiyu1002 commented 2 years ago

提供一个可以复现的 demo 或者 贴下 暴露dubbo 服务，引用dubbo服务 相关的代码 以及对应的接口实现

cwt9562 commented 2 years ago

我找到并规避了这个问题，但我觉得，它应该是可以被解决了 provider错乱的原因是，我自定义了一个comsumer的filter 这个filter主要任务是，将当前登录用户（登录由shiro管理）中的userid，传递给privider侧，用来记录类似操作日志、变更日志一类的东西

@Activate(group = CommonConstants.CONSUMER)
@Slf4j
public class CustomContextFilter implements Filter {

    @Override
    public Result invoke(Invoker<?> invoker, Invocation invocation) throws RpcException {
        setCurrentUserId();
        setCurrentRemoteIp();
        return invoker.invoke(invocation);
    }

    private void setCurrentUserId() {
        ...
        RpcContext.getClientAttachment().setAttachment(Constants.CURRENT_USER_ID, currentUserId);
        ...
    }

    private void setCurrentRemoteIp() {
        ...
        RpcContext.getClientAttachment().setAttachment(Constants.CURRENT_REMOTE_IP, currentRemoteIp);
        ...
    }

}

但是，我在setCurrentUserId()方法里，犯了一个错误， 层层调用之下，居然发起了另一个dubbo rpc请求，去调用了UserProvider的getByUsername()方法 在UserProvider的getByUsername()被调用完毕后，回到CustomContextFilter的invoke()方法的return invoker.invoke(invocation);后，原本的调用就错乱了 不管本次请求原本是调用的AbcProvider的abc()方法，还是XyzProvider的xyz()方法 统统变成了UserProvider的abc()方法和UserProvider的xyz()方法

cwt9562 commented 2 years ago

然后，我规避这个问题的方法就是，在filter里，不要触发其他rpc调用

cwt9562 commented 2 years ago

然后我说的，它应该可以被解决 是说，要么dubbo引擎应该类似递归算法一般，支持rpc的嵌套 比如像我的这个场景，在filter里发起另一个rpc 要么检测到有这个行为时，报特定的错误，加以制止 而不是真的将错乱的请求，发送到provider侧

AlbumenJ commented 2 years ago

在filter里发起另一个rpc

这个本质上是支持的，但是需要调用方自己评估影响面，主要是 ThreadLocal 相关的数据

cwt9562 commented 2 years ago

@liufeiyu1002 @AlbumenJ @lixiaojiee https://github.com/cwt9562/dubbo-privider-confusion 我基于官方的spring boot demo， 复现了 你们看下

cwt9562 commented 2 years ago

启动provider后， 再启动comsumer 就会得到 Caused by: org.apache.dubbo.common.bytecode.NoSuchMethodException: Not found method "sayHello" in class org.apache.dubbo.springboot.demo.provider.AnotherServiceImpl.

lixiaojiee commented 2 years ago

@liufeiyu1002 @AlbumenJ @lixiaojiee https://github.com/cwt9562/dubbo-privider-confusion 我基于官方的spring boot demo， 复现了 你们看下 收到，我看下

lixiaojiee commented 2 years ago

启动provider后， 再启动comsumer 就会得到 Caused by: org.apache.dubbo.common.bytecode.NoSuchMethodException: Not found method "sayHello" in class org.apache.dubbo.springboot.demo.provider.AnotherServiceImpl.

看了下，出现问题是由于您自定义的一个拦截器里又调了一次dubbo方法，如下图：

image

根本问题是在调用这个方法结束后，RpcContext里面的SERVICE_CONTEXT没有做清理，导致在继续执行调用DemoService的sayHello方法客户端逻辑时读取了已经过期的缓存（attachments.path=org.apache.dubbo.springboot.demo.AnotherService），而服务端正是通过这个path值定位服务端骨架的。感兴趣的话，您可以提个pr修复这个问题。

cwt9562 commented 2 years ago

启动provider后， 再启动comsumer 就会得到 Caused by: org.apache.dubbo.common.bytecode.NoSuchMethodException: Not found method "sayHello" in class org.apache.dubbo.springboot.demo.provider.AnotherServiceImpl.

看了下，出现问题是由于您自定义的一个拦截器里又调了一次dubbo方法，如下图： image 根本问题是在调用这个方法结束后，RpcContext里面的SERVICE_CONTEXT没有做清理，导致在继续执行调用DemoService的sayHello方法客户端逻辑时读取了已经过期的缓存（attachments.path=org.apache.dubbo.springboot.demo.AnotherService），而服务端正是通过这个path值定位服务端骨架的。感兴趣的话，您可以提个pr修复这个问题。

我对源码真不熟悉，哈哈哈，提不了 我的demo能帮忙定位到问题就好

AlbumenJ commented 2 years ago

RpcContext里面的SERVICE_CONTEXT没有做清理

可以在 org.apache.dubbo.rpc.proxy.InvocationUtil#invoke 里面保存并清理下 SERVICE_CONTEXT，请求结束后还原

lixiaojiee commented 2 years ago

我想了下， @cwt9562 实现的这个过滤器本身可能就是有问题的，在过滤器里再进行RPC请求，弄不好会把系统搞死，我们是否需要针对这种场景做下控制，禁止在框架内做嵌套RPC调用？

lixiaojiee commented 2 years ago

我想了下， @cwt9562 实现的这个过滤器本身可能就是有问题的，在过滤器里再进行RPC请求，弄不好会把系统搞死，我们是否需要针对这种场景做下控制，禁止在框架内做嵌套RPC调用？

@AlbumenJ

AlbumenJ commented 2 years ago

我想了下， @cwt9562 实现的这个过滤器本身可能就是有问题的，在过滤器里再进行RPC请求，弄不好会把系统搞死，我们是否需要针对这种场景做下控制，禁止在框架内做嵌套RPC调用？

嵌套调用是有可能有这个需求的，比如请求转发就需要使用到嵌套调用。框架侧来说应该也是可以保证请求成功的（处理好 ThreadLocal 之后）

lixiaojiee commented 2 years ago

我想了下， @cwt9562 实现的这个过滤器本身可能就是有问题的，在过滤器里再进行RPC请求，弄不好会把系统搞死，我们是否需要针对这种场景做下控制，禁止在框架内做嵌套RPC调用？

嵌套调用是有可能有这个需求的，比如请求转发就需要使用到嵌套调用。框架侧来说应该也是可以保证请求成功的（处理好 ThreadLocal 之后）

明白，那这样的话，除了你说的问题需要处理好，还需要框架测处理死循环的问题。我这周试试改一下。

AlbumenJ commented 2 years ago

我想了下， @cwt9562 实现的这个过滤器本身可能就是有问题的，在过滤器里再进行RPC请求，弄不好会把系统搞死，我们是否需要针对这种场景做下控制，禁止在框架内做嵌套RPC调用？

嵌套调用是有可能有这个需求的，比如请求转发就需要使用到嵌套调用。框架侧来说应该也是可以保证请求成功的（处理好 ThreadLocal 之后）

明白，那这样的话，除了你说的问题需要处理好，还需要框架测处理死循环的问题。我这周试试改一下。

Track in #10766