mrniko / netty-socketio

Socket.IO server implemented on Java. Realtime java framework
Apache License 2.0
6.82k stars 1.65k forks source link

netty LEAK: ByteBuf.release() was not called before it's garbage-collected. #173

Closed evgeny-pasynkov closed 9 years ago

evgeny-pasynkov commented 10 years ago

Hi, netty with "ResourceLeakDetector.setLevel(ResourceLeakDetector.Level.ADVANCED);" regularry shows the following error:

[2014-10-15 02:38:12,872] ERROR socket listener-1-1 etty.util.ResourceLeakDetector - error LEAK: ByteBuf.release() was not called before it's garbage-collected. Recent access records: 2

2:

io.netty.buffer.AdvancedLeakAwareByteBuf.writeByte(AdvancedLeakAwareByteBuf.java:499)
com.corundumstudio.socketio.protocol.PacketEncoder.encodePacket(PacketEncoder.java:230)
com.corundumstudio.socketio.protocol.PacketEncoder.encodePackets(PacketEncoder.java:112)
com.corundumstudio.socketio.handler.EncoderHandler.handleHTTP(EncoderHandler.java:244)
com.corundumstudio.socketio.handler.EncoderHandler.write(EncoderHandler.java:194)
io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:658)
io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:716)
io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:706)
io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:741)
io.netty.channel.DefaultChannelPipeline.writeAndFlush(DefaultChannelPipeline.java:895)
io.netty.channel.AbstractChannel.writeAndFlush(AbstractChannel.java:240)
com.corundumstudio.socketio.handler.ClientHead.sendPackets(ClientHead.java:149)
com.corundumstudio.socketio.handler.ClientHead.bindChannel(ClientHead.java:104)
com.corundumstudio.socketio.transport.PollingTransport.onGet(PollingTransport.java:172)
com.corundumstudio.socketio.transport.PollingTransport.handleMessage(PollingTransport.java:122)
com.corundumstudio.socketio.transport.PollingTransport.channelRead(PollingTransport.java:97)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
com.corundumstudio.socketio.handler.AuthorizeHandler.channelRead(AuthorizeHandler.java:115)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
com.jetbrains.upsource.netty.handlers.SimpleChannelInboundHandlerEx.channelRead(SimpleChannelInboundHandlerEx.java:73)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
com.jetbrains.upsource.netty.handlers.SimpleChannelInboundHandlerEx.channelRead(SimpleChannelInboundHandlerEx.java:73)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:163)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:130)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
java.lang.Thread.run(Thread.java:745)

1:

io.netty.buffer.AdvancedLeakAwareByteBuf.writeByte(AdvancedLeakAwareByteBuf.java:499)
com.corundumstudio.socketio.protocol.PacketEncoder.encodePacket(PacketEncoder.java:209)
com.corundumstudio.socketio.protocol.PacketEncoder.encodePackets(PacketEncoder.java:112)
com.corundumstudio.socketio.handler.EncoderHandler.handleHTTP(EncoderHandler.java:244)
com.corundumstudio.socketio.handler.EncoderHandler.write(EncoderHandler.java:194)
io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:658)
io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:716)
io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:706)
io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:741)
io.netty.channel.DefaultChannelPipeline.writeAndFlush(DefaultChannelPipeline.java:895)
io.netty.channel.AbstractChannel.writeAndFlush(AbstractChannel.java:240)
com.corundumstudio.socketio.handler.ClientHead.sendPackets(ClientHead.java:149)
com.corundumstudio.socketio.handler.ClientHead.bindChannel(ClientHead.java:104)
com.corundumstudio.socketio.transport.PollingTransport.onGet(PollingTransport.java:172)
com.corundumstudio.socketio.transport.PollingTransport.handleMessage(PollingTransport.java:122)
com.corundumstudio.socketio.transport.PollingTransport.channelRead(PollingTransport.java:97)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
com.corundumstudio.socketio.handler.AuthorizeHandler.channelRead(AuthorizeHandler.java:115)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
com.jetbrains.upsource.netty.handlers.SimpleChannelInboundHandlerEx.channelRead(SimpleChannelInboundHandlerEx.java:73)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
com.jetbrains.upsource.netty.handlers.SimpleChannelInboundHandlerEx.channelRead(SimpleChannelInboundHandlerEx.java:73)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:163)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:130)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
java.lang.Thread.run(Thread.java:745)

Created at: io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:55) io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155) io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:146) io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:99) com.corundumstudio.socketio.protocol.PacketEncoder.allocateBuffer(PacketEncoder.java:51) com.corundumstudio.socketio.protocol.PacketEncoder.encodePacket(PacketEncoder.java:206) com.corundumstudio.socketio.protocol.PacketEncoder.encodePackets(PacketEncoder.java:112) com.corundumstudio.socketio.handler.EncoderHandler.handleHTTP(EncoderHandler.java:244) com.corundumstudio.socketio.handler.EncoderHandler.write(EncoderHandler.java:194) io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:658) io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:716) io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:706) io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:741) io.netty.channel.DefaultChannelPipeline.writeAndFlush(DefaultChannelPipeline.java:895) io.netty.channel.AbstractChannel.writeAndFlush(AbstractChannel.java:240) com.corundumstudio.socketio.handler.ClientHead.sendPackets(ClientHead.java:149) com.corundumstudio.socketio.handler.ClientHead.bindChannel(ClientHead.java:104) com.corundumstudio.socketio.transport.PollingTransport.onGet(PollingTransport.java:172) com.corundumstudio.socketio.transport.PollingTransport.handleMessage(PollingTransport.java:122) com.corundumstudio.socketio.transport.PollingTransport.channelRead(PollingTransport.java:97) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319) com.corundumstudio.socketio.handler.AuthorizeHandler.channelRead(AuthorizeHandler.java:115) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319) com.jetbrains.upsource.netty.handlers.SimpleChannelInboundHandlerEx.channelRead(SimpleChannelInboundHandlerEx.java:73) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319) io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319) com.jetbrains.upsource.netty.handlers.SimpleChannelInboundHandlerEx.channelRead(SimpleChannelInboundHandlerEx.java:73) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319) io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319) io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:163) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319) io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787) io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:130) io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116) java.lang.Thread.run(Thread.java:745)

mrniko commented 10 years ago

Please check it and let me know if it works

evgeny-pasynkov commented 10 years ago

No, the problem is not fixed. I just wonder how your change could fix it...

You allocate buffer with lines: if (!binary) { buf = allocateBuffer(allocator); }

And, I suppose, it should be released with something like: if (!binary) { .......... buffer.writeBytes(buf); buf.release(); //!!!!!!! }

Isn't it?

mrniko commented 10 years ago

You don't need release a buffer which has been already transferred to another by writeBytes method. Here is an example of this case EncoderHandler.handleWebsocket

mrniko commented 10 years ago

Maybe you have an exceptions in this code block which may lead to unreleased allocated buffer?

evgeny-pasynkov commented 10 years ago

Ok, I see that netty code above could sometimes silently ignore exceptions. I've modified locally your code to log all exceptions in the PacketEncoder class, and will be back if something will be catched and logged

evgeny-pasynkov commented 10 years ago

Negative. No exceptions, and the leak remains. BTW, I haven't nowhere found in the netty docs/sources that "writeBytes" method releases the buffer, and the mentioned above "EncoderHandler.handleWebsocket" releases it explicitly

evgeny-pasynkov commented 10 years ago

So, I've locally added "buf.release()", and have never seen this problem after that. Looks like this line makes the business.