search

dangdangdotcom / dubbox

Dubbox now means Dubbo eXtensions, and it adds features like RESTful remoting, Kyro/FST serialization, etc to the Dubbo service framework.
http://dangdangdotcom.github.io/dubbox
Apache License 2.0
4.89k stars 2.06k forks source link

kryo 序列化 null 的问题~ #356

Open Gaion opened 7 years ago

Gaion commented 7 years ago

使用过程中，发现一个问题，直连情况下参数传 null 正常，但是从注册中心走一遍，报错~

api

public interface DemoService {
    void show(Long a, Long b, String c);
}

提供方

public class DemoServiceImpl implements DemoService {
    public void show(Long a, Long b, String c) {
        System.out.println(StringUtils.format("a:{};b:{},c:{}", a, b, c));
    }
}
    <dubbo:application name="bsm-support"/>
    <dubbo:protocol name="dubbo" port="20880" serialization="kryo"/>
    <dubbo:registry address="zookeeper://127.0.0.1:2181" />
    <dubbo:annotation package="com.demo.bsm.support" />
    <dubbo:provider timeout="10000" retries="0"/>

消费者

public void show() {
    demoService.show(1L, 2L, "c");
    demoService.show(1L, 2L, null);
    demoService.show(1L, null, "c");
}

消费者直连XML

    <dubbo:application name="supervise-web"/>
    <dubbo:consumer registry="N/A" timeout="10000"/>
    <dubbo:annotation package="com.demo.supervise.web" />

调用后提供方输出

a:1;b:2,c:c a:1;b:2,c: a:1;b:,c:c

消费者通过注册中心的XML

    <dubbo:application name="supervise-web"/>
    <dubbo:registry address="zookeeper://127.0.0.1:2181"/>
    <dubbo:annotation package="com.demo.supervise.web" />

调用后提供方输出

a:1;b:2,c:c

提供方报错

15:44:13.340 [New I/O worker #1] WARN com.alibaba.dubbo.rpc.protocol.dubbo.DecodeableRpcInvocation - [DUBBO] Decode rpc invocation failed: java.lang.String cannot be cast to java.util.Map, dubbo version: 2.8.4, current host: 10.37.129.2 java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map at com.alibaba.dubbo.rpc.protocol.dubbo.DecodeableRpcInvocation.decode(DecodeableRpcInvocation.java:150) ~[dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.rpc.protocol.dubbo.DecodeableRpcInvocation.decode(DecodeableRpcInvocation.java:74) [dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.rpc.protocol.dubbo.DubboCodec.decodeBody(DubboCodec.java:138) [dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.remoting.exchange.codec.ExchangeCodec.decode(ExchangeCodec.java:134) [dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.remoting.exchange.codec.ExchangeCodec.decode(ExchangeCodec.java:95) [dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.rpc.protocol.dubbo.DubboCountCodec.decode(DubboCountCodec.java:46) [dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.remoting.transport.netty.NettyCodecAdapter$InternalDecoder.messageReceived(NettyCodecAdapter.java:134) [dubbo-2.8.4.jar:2.8.4] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [netty-3.7.0.Final.jar:?] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.7.0.Final.jar:?] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) [netty-3.7.0.Final.jar:?] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) [netty-3.7.0.Final.jar:?] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) [netty-3.7.0.Final.jar:?] at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) [netty-3.7.0.Final.jar:?] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) [netty-3.7.0.Final.jar:?] at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) [netty-3.7.0.Final.jar:?] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) [netty-3.7.0.Final.jar:?] at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) [netty-3.7.0.Final.jar:?] at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) [netty-3.7.0.Final.jar:?] at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) [netty-3.7.0.Final.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [?:1.7.0_75] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [?:1.7.0_75] at java.lang.Thread.run(Thread.java:745) [?:1.7.0_75]

消费者报错

15:16:29.392 [http-bio-8081-exec-1] ERROR com.demo.common.bean.web.ExceptionHandler - Failed to invoke the method show in the service com.demo.bsm.support.api.service.DemoService. Tried 1 times of the providers [10.37.129.2:20880] (1/1) from the registry 127.0.0.1:2181 on the consumer 10.37.129.2 using the dubbo version 2.8.4. Last error is: Failed to invoke remote method: show, provider: dubbo://10.37.129.2:20880/com.demo.bsm.support.api.service.DemoService?anyhost=true&application=supervise-web&check=false&default.retries=0&default.timeout=10000&dubbo=2.8.4&generic=false&interface=com.demo.bsm.support.api.service.DemoService&methods=show&pid=3686&serialization=kryo&side=consumer&timestamp=1504077312779, cause: Fail to decode request due to: RpcInvocation [methodName=show, parameterTypes=[], arguments=null, attachments={dubbo=2.8.4, input=242, path=com.demo.bsm.support.api.service.DemoService, version=0.0.0}] com.alibaba.dubbo.rpc.RpcException: Failed to invoke the method show in the service com.demo.bsm.support.api.service.DemoService. Tried 1 times of the providers [10.37.129.2:20880] (1/1) from the registry 127.0.0.1:2181 on the consumer 10.37.129.2 using the dubbo version 2.8.4. Last error is: Failed to invoke remote method: show, provider: dubbo://10.37.129.2:20880/com.demo.bsm.support.api.service.DemoService?anyhost=true&application=supervise-web&check=false&default.retries=0&default.timeout=10000&dubbo=2.8.4&generic=false&interface=com.demo.bsm.support.api.service.DemoService&methods=show&pid=3686&serialization=kryo&side=consumer&timestamp=1504077312779, cause: Fail to decode request due to: RpcInvocation [methodName=show, parameterTypes=[], arguments=null, attachments={dubbo=2.8.4, input=242, path=com.demo.bsm.support.api.service.DemoService, version=0.0.0}] at com.alibaba.dubbo.rpc.cluster.support.FailoverClusterInvoker.doInvoke(FailoverClusterInvoker.java:101) ~[dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.rpc.cluster.support.AbstractClusterInvoker.invoke(AbstractClusterInvoker.java:227) ~[dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.rpc.cluster.support.wrapper.MockClusterInvoker.invoke(MockClusterInvoker.java:72) ~[dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.rpc.proxy.InvokerInvocationHandler.invoke(InvokerInvocationHandler.java:52) ~[dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.common.bytecode.proxy2.show(proxy2.java) ~[dubbo-2.8.4.jar:2.8.4] at com.demo.supervise.web.service.ShopService.show(ShopService.java:36) ~[ShopService.class:?] at com.demo.supervise.web.controller.ShopController.regionDialog(ShopController.java:151) ~[ShopController.class:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_75] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_75] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_75] at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_75] at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:220) ~[spring-web-4.3.5.RELEASE.jar:4.3.5.RELEASE] at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:134) ~[spring-web-4.3.5.RELEASE.jar:4.3.5.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:116) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE] at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE] at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE] at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963) [spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE] at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897) [spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE] at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) [spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE] at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861) [spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE] at javax.servlet.http.HttpServlet.service(HttpServlet.java:620) [servlet-api.jar:?] at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) [spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE] at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) [servlet-api.jar:?] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) [catalina.jar:7.0.57] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.57] at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat7-websocket.jar:7.0.57] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.57] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.57] at org.springframework.orm.jpa.support.OpenEntityManagerInViewFilter.doFilterInternal(OpenEntityManagerInViewFilter.java:178) [spring-orm-4.3.5.RELEASE.jar:4.3.5.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.5.RELEASE.jar:4.3.5.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.57] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.57] at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61) [shiro-web-1.2.4.jar:1.2.4] at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) [shiro-web-1.2.4.jar:1.2.4] at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) [shiro-web-1.2.4.jar:1.2.4] at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) [shiro-web-1.2.4.jar:1.2.4] at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) [shiro-web-1.2.4.jar:1.2.4] at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) [shiro-web-1.2.4.jar:1.2.4] at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) [shiro-web-1.2.4.jar:1.2.4] at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) [shiro-core-1.2.4.jar:1.2.4] at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) [shiro-core-1.2.4.jar:1.2.4] at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) [shiro-core-1.2.4.jar:1.2.4] at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) [shiro-web-1.2.4.jar:1.2.4] at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) [shiro-web-1.2.4.jar:1.2.4] at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) [spring-web-4.3.5.RELEASE.jar:4.3.5.RELEASE] at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262) [spring-web-4.3.5.RELEASE.jar:4.3.5.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.57] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.57] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) [catalina.jar:7.0.57] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) [catalina.jar:7.0.57] at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) [catalina.jar:7.0.57] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) [catalina.jar:7.0.57] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) [catalina.jar:7.0.57] at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) [catalina.jar:7.0.57] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) [catalina.jar:7.0.57] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) [catalina.jar:7.0.57] at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) [tomcat-coyote.jar:7.0.57] at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) [tomcat-coyote.jar:7.0.57] at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) [tomcat-coyote.jar:7.0.57] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [?:1.7.0_75] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [?:1.7.0_75] at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-coyote.jar:7.0.57] at java.lang.Thread.run(Thread.java:745) [?:1.7.0_75] Caused by: com.alibaba.dubbo.remoting.RemotingException: Fail to decode request due to: RpcInvocation [methodName=show, parameterTypes=[], arguments=null, attachments={dubbo=2.8.4, input=242, path=com.demo.bsm.support.api.service.DemoService, version=0.0.0}] at com.alibaba.dubbo.remoting.exchange.support.DefaultFuture.returnFromResponse(DefaultFuture.java:190) ~[dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.remoting.exchange.support.DefaultFuture.get(DefaultFuture.java:110) ~[dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.remoting.exchange.support.DefaultFuture.get(DefaultFuture.java:84) ~[dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.rpc.protocol.dubbo.DubboInvoker.doInvoke(DubboInvoker.java:96) ~[dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.rpc.protocol.AbstractInvoker.invoke(AbstractInvoker.java:144) ~[dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.rpc.listener.ListenerInvokerWrapper.invoke(ListenerInvokerWrapper.java:74) ~[dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.rpc.protocol.dubbo.filter.FutureFilter.invoke(FutureFilter.java:53) ~[dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:91) ~[dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.monitor.support.MonitorFilter.invoke(MonitorFilter.java:75) ~[dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:91) ~[dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.rpc.filter.ConsumerContextFilter.invoke(ConsumerContextFilter.java:48) ~[dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:91) ~[dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.rpc.protocol.InvokerWrapper.invoke(InvokerWrapper.java:53) ~[dubbo-2.8.4.jar:2.8.4] at com.alibaba.dubbo.rpc.cluster.support.FailoverClusterInvoker.doInvoke(FailoverClusterInvoker.java:77) ~[dubbo-2.8.4.jar:2.8.4] ... 63 more

Gaion commented 7 years ago

翻了半天源码，发现，使用 kyro、fst 进行序列化时，参数为 null 的情况下，就不写入参数的数量，导致在反序列化的时候由于获取不到参数数量而报错，默认的序列化访问正常。问一下，原来为啥设计成这样？是 BUG，还是有特殊考虑？

序列化时的代码，com.alibaba.dubbo.rpc.protocol.dubbo.DubboCodec的encodeRequestData方法

protected void encodeRequestData(Channel channel, ObjectOutput out, Object data) throws IOException {
    .....
    if (getSerialization(channel) instanceof OptimizedSerialization) {  //kyro、fst序列化
        if (!containComplexArguments(inv)) { //写入参数数量
            out.writeInt(inv.getParameterTypes().length);
        } else { //无参数
            out.writeInt(-1);
        }
    } else { //其他序列化方式
        out.writeUTF(ReflectUtils.getDesc(inv.getParameterTypes()));
    }
    .....
}
private boolean containComplexArguments(RpcInvocation invocation) {
    for (int i = 0; i < invocation.getParameterTypes().length; i++) {
        if (invocation.getArguments()[i] == null || invocation.getParameterTypes()[i] != invocation.getArguments()[i].getClass()) { //参数为 null，或者实际参数类型与定义不一致？
            return true;
        }
    }
    return false;
}

反序列化时的代码，com.alibaba.dubbo.rpc.protocol.dubbo.DecodeableRpcInvocation 的 decode 方法

public Object decode(Channel channel, InputStream input) throws IOException {
    .....
    int argNum = -1;
    if (CodecSupport.getSerialization(channel.getUrl(), serializationType) instanceof OptimizedSerialization) { //kyro、fst 序列化时，读取参数数量
        argNum = in.readInt();
    }
    if (argNum >= 0) { // 包含参数的处理
        ...
    } else { //没有读取到参数，则认为是其他序列化
        ...
    }
    ...
}