search

apache / dubbo

The java implementation of Apache Dubbo. An RPC and microservice framework.
https://dubbo.apache.org/
Apache License 2.0
40.48k stars 26.43k forks source link

Argument of Roaring64NavigableMap type threw exception #10071

Closed chenyongyin closed 2 years ago

chenyongyin commented 2 years ago

version: 2.7.7 Return Roaring64NavigableMap throws an exception：

threw exception [Request processing failed; nested exception is org.apache.dubbo.rpc.RpcException: Failed to invoke the method getMemberData in the service Tried 3 times of the providers [xxxx] (1/1) from the registry xxx on the consumer xxxx using the dubbo version 2.7.7. Last error is: Failed to invoke remote method: xxxx, provider,cause: org.apache.dubbo.remoting.RemotingException: Failed to send response: Response [id=4, version=2.0.2, status=20, event=false, error=null, result=AppResponse [value={0,682,684,685,719,720,721,722,725,726,728,729,752,759,771,772,776,782,793,796,1730,1811,1833,1952,1965,1968,1970,1971,1972,1975,1976,1977,1978,1979,1981,1984,1987,1988,1989,1990,1991,1993,2000,2001}, exception=null]], cause: java.lang.RuntimeException: Serialized class org.roaringbitmap.RoaringBitmapSupplier must implement java.io.Serializable Java field: private org.roaringbitmap.BitmapDataProviderSupplier org.roaringbitmap.longlong.Roaring64NavigableMap.supplier java.lang.RuntimeException: Serialized class org.roaringbitmap.RoaringBitmapSupplier must implement java.io.Serializable Java field: private org.roaringbitmap.BitmapDataProviderSupplier org.roaringbitmap.longlong.Roaring64NavigableMap.supplier at com.alibaba.com.caucho.hessian.io.JavaSerializer$FieldSerializer.serialize(JavaSerializer.java:304) at com.alibaba.com.caucho.hessian.io.JavaSerializer.writeInstance(JavaSerializer.java:284) at com.alibaba.com.caucho.hessian.io.JavaSerializer.writeObject(JavaSerializer.java:251) at com.alibaba.com.caucho.hessian.io.Hessian2Output.writeObject(Hessian2Output.java:412) at org.apache.dubbo.common.serialize.hessian2.Hessian2ObjectOutput.writeObject(Hessian2ObjectOutput.java:97) at org.apache.dubbo.rpc.protocol.dubbo.DubboCodec.encodeResponseData(DubboCodec.java:203) at org.apache.dubbo.remoting.exchange.codec.ExchangeCodec.encodeResponse(ExchangeCodec.java:283) at org.apache.dubbo.remoting.exchange.codec.ExchangeCodec.encode(ExchangeCodec.java:71) at org.apache.dubbo.rpc.protocol.dubbo.DubboCountCodec.encode(DubboCountCodec.java:40) at org.apache.dubbo.remoting.transport.netty4.NettyCodecAdapter$InternalEncoder.encode(NettyCodecAdapter.java:69) at io.netty.handler.codec.MessageToByteEncoder.write(MessageToByteEncoder.java:107) at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:717) at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:709) at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:792) at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:702) at io.netty.handler.timeout.IdleStateHandler.write(IdleStateHandler.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:717) at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:709) at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:792) at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:702) at io.netty.channel.ChannelDuplexHandler.write(ChannelDuplexHandler.java:115) at org.apache.dubbo.remoting.transport.netty4.NettyServerHandler.write(NettyServerHandler.java:104) at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:717) at io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:764) at io.netty.channel.AbstractChannelHandlerContext$WriteTask.run(AbstractChannelHandlerContext.java:1071) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:164) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.IllegalStateException: Serialized class org.roaringbitmap.RoaringBitmapSupplier must implement java.io.Serializable at com.alibaba.com.caucho.hessian.io.SerializerFactory.getDefaultSerializer(SerializerFactory.java:402) at com.alibaba.com.caucho.hessian.io.SerializerFactory.getSerializer(SerializerFactory.java:376) at com.alibaba.com.caucho.hessian.io.Hessian2Output.writeObject(Hessian2Output.java:411) at com.alibaba.com.caucho.hessian.io.JavaSerializer$FieldSerializer.serialize(JavaSerializer.java:302) ... 32 more ] with root cause org.apache.dubbo.remoting.RemotingException: Failed to send response: Response [id=4, version=2.0.2, status=20, event=false, error=null, result=AppResponse [value={0,682,684,685,719,720,721,722,725,726,728,729,752,759,771,772,776,782,793,796,1730,1811,1833,1952,1965,1968,1970,1971,1972,1975,1976,1977,1978,1979,1981,1984,1987,1988,1989,1990,1991,1993,2000,2001}, exception=null]], cause: java.lang.RuntimeException: Serialized class org.roaringbitmap.RoaringBitmapSupplier must implement java.io.Serializable Java field: private org.roaringbitmap.BitmapDataProviderSupplier org.roaringbitmap.longlong.Roaring64NavigableMap.supplier java.lang.RuntimeException: Serialized class org.roaringbitmap.RoaringBitmapSupplier must implement java.io.Serializable Java field: private org.roaringbitmap.BitmapDataProviderSupplier org.roaringbitmap.longlong.Roaring64NavigableMap.supplier at com.alibaba.com.caucho.hessian.io.JavaSerializer$FieldSerializer.serialize(JavaSerializer.java:304) at com.alibaba.com.caucho.hessian.io.JavaSerializer.writeInstance(JavaSerializer.java:284) at com.alibaba.com.caucho.hessian.io.JavaSerializer.writeObject(JavaSerializer.java:251) at com.alibaba.com.caucho.hessian.io.Hessian2Output.writeObject(Hessian2Output.java:412) at org.apache.dubbo.common.serialize.hessian2.Hessian2ObjectOutput.writeObject(Hessian2ObjectOutput.java:97) at org.apache.dubbo.rpc.protocol.dubbo.DubboCodec.encodeResponseData(DubboCodec.java:203) at org.apache.dubbo.remoting.exchange.codec.ExchangeCodec.encodeResponse(ExchangeCodec.java:283) at org.apache.dubbo.remoting.exchange.codec.ExchangeCodec.encode(ExchangeCodec.java:71) at org.apache.dubbo.rpc.protocol.dubbo.DubboCountCodec.encode(DubboCountCodec.java:40) at org.apache.dubbo.remoting.transport.netty4.NettyCodecAdapter$InternalEncoder.encode(NettyCodecAdapter.java:69) at io.netty.handler.codec.MessageToByteEncoder.write(MessageToByteEncoder.java:107) at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:717) at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:709) at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:792) at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:702) at io.netty.handler.timeout.IdleStateHandler.write(IdleStateHandler.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:717) at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:709) at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:792) at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:702) at io.netty.channel.ChannelDuplexHandler.write(ChannelDuplexHandler.java:115) at org.apache.dubbo.remoting.transport.netty4.NettyServerHandler.write(NettyServerHandler.java:104) at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:717) at io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:764) at io.netty.channel.AbstractChannelHandlerContext$WriteTask.run(AbstractChannelHandlerContext.java:1071) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:164) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.IllegalStateException: Serialized class org.roaringbitmap.RoaringBitmapSupplier must implement java.io.Serializable at com.alibaba.com.caucho.hessian.io.SerializerFactory.getDefaultSerializer(SerializerFactory.java:402) at com.alibaba.com.caucho.hessian.io.SerializerFactory.getSerializer(SerializerFactory.java:376) at com.alibaba.com.caucho.hessian.io.Hessian2Output.writeObject(Hessian2Output.java:411) at com.alibaba.com.caucho.hessian.io.JavaSerializer$FieldSerializer.serialize(JavaSerializer.java:302) ... 32 more

kimmking commented 2 years ago

Serialized class org.roaringbitmap.RoaringBitmapSupplier must implement java.io.Serializable

chenyongyin commented 2 years ago

Serialized class org.roaringbitmap.RoaringBitmapSupplier must implement java.io.Serializable

Roaring64NavigableMap implements Externalizable,Roaring64NavigableMap.writeExternal does not serialize RoaringBitmapSupplier field, so RoaringBitmapSupplier Don't need implements java.io.Serializable

lemire commented 2 years ago

Reference: https://github.com/RoaringBitmap/RoaringBitmap/issues/563

lemire commented 2 years ago

Kryo should work.

lemire commented 2 years ago

I recommend against serializing Roaring bitmaps using the Hessian protocol.

chickenlj commented 2 years ago

Hi @chenyongyin, I don't know the structure of Roaring64NavigableMap and why you need to serialize it.

java.lang.RuntimeException: Serialized class org.roaringbitmap.RoaringBitmapSupplier must implement java.io.Serializable
Java field: private org.roaringbitmap.BitmapDataProviderSupplier

But the exception shows that this class cannot work together with hessian2 well. So you need to think of other ways to achieve that, for example, using some other serialization protocols supported by Dubbo.

kimmking commented 1 year ago

https://github.com/RoaringBitmap/RoaringBitmap/issues/563#issuecomment-1309702279