search

lightbody / browsermob-proxy

A free utility to help web developers watch and manipulate network traffic from their AJAX applications.
http://bmp.lightbody.net
Apache License 2.0
2.17k stars 659 forks source link

BrowserMob proxy keeps throwing io.netty.util.IllegalReferenceCountException #789

Closed danishS closed 5 years ago

danishS commented 6 years ago

I'm using Browsermob proxy (BMP) with Watir implementation. I have the BMP server and proxy setup properly, i'm routing my selenium traffic through the proxy correctly as well, however, when I run the tests, I notice the server logs keeps throwing the io.netty.util.IllegalReferenceCountException. Not sure why is this happening or how to fix it. I'm setting up BrowserMob Proxy like so (for brevity not including full code)

class BrowserMobProxy
   class << self
     def config
      @server = BrowserMob::Proxy::Server.new("path/to/bmp/binary")
      @server.start
      @proxy = @server.create_proxy
      @proxy.selenium_proxy
    end
    def run_capture
      @proxy.new_har(capture_content: true)
          yield
      @har = @proxy.har
    end
  def save_run
    @har.save_to <path/to/harfile>
   end
  def turn_off
     @proxy.close
     @server.stop
  end
end

In my browser startup, i'm doing the following, it is a bit brute forced, because it wasn't working any other way, i.e. the capabilities weren't getting set with proxy object

driver = Selenium::WebDriver.for :chrome, desired_capabilities: <passed in proxy objet>
Watir::Browser.new driver

Then in my cucumber test I have the following

Then("I get to a webpage and perform a task") do
   BrowserMobProxy.run_capture do
     on(MyPage) do |page|
       //some code
       BrowserMobProxy.save_run
     end
  end
end

Following is the trace

[INFO  2018-11-07T20:59:28,811 org.eclipse.jetty.util.log] (main) jetty-7.x.y-SNAPSHOT 
[INFO  2018-11-07T20:59:28,896 org.eclipse.jetty.util.log] (main) started o.e.j.s.ServletContextHandler{/,null} 
[INFO  2018-11-07T20:59:29,246 org.eclipse.jetty.util.log] (main) Started SelectChannelConnector@0.0.0.0:9090 
[INFO  2018-11-07T20:59:30,761 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp1969856053-18) Starting proxy at address: 0.0.0.0/0.0.0.0:8081 
[INFO  2018-11-07T20:59:30,803 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp1969856053-18) Proxy listening with TCP transport 
[INFO  2018-11-07T20:59:31,026 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (qtp1969856053-18) Proxy started at address: /0:0:0:0:0:0:0:0:8081 
[WARN  2018-11-07T20:59:39,798 io.netty.util.ReferenceCountUtil] (LittleProxy-0-ClientToProxyWorker-3) Failed to release a message: PooledSlicedByteBuf(freed) io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
    at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:110) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.buffer.AbstractPooledDerivedByteBuf.deallocate(AbstractPooledDerivedByteBuf.java:73) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:115) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:98) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:84) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.util.ReferenceCountUtil.safeRelease(ReferenceCountUtil.java:109) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.PendingWriteQueue.removeAndFailAll(PendingWriteQueue.java:171) [browsermob-dist-2.1.4.jar:?]
    at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:493) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.access$1500(AbstractChannelHandlerContext.java:39) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:1148) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:1089) [browsermob-dist-2.1.4.jar:?]
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) [browsermob-dist-2.1.4.jar:?]
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454) [browsermob-dist-2.1.4.jar:?]
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) [browsermob-dist-2.1.4.jar:?]
    at java.lang.Thread.run(Thread.java:844) [?:?]

[ERROR 2018-11-07T20:59:39,846 org.littleshoot.proxy.impl.ClientToProxyConnection] (LittleProxy-0-ClientToProxyWorker-3) (AWAITING_INITIAL) [id: 0x0e21a916, L:/0:0:0:0:0:0:0:1:8081 - R:/0:0:0:0:0:0:0:1:61133]: Caught an exception on ClientToProxyConnection io.netty.util.IllegalReferenceCountException: refCnt: 0
    at io.netty.buffer.AbstractByteBuf.ensureAccessible(AbstractByteBuf.java:1407) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1353) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.buffer.PooledUnsafeDirectByteBuf.nioBuffer(PooledUnsafeDirectByteBuf.java:324) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.buffer.PooledSlicedByteBuf.nioBuffer(PooledSlicedByteBuf.java:87) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.buffer.AbstractPooledDerivedByteBuf.internalNioBuffer(AbstractPooledDerivedByteBuf.java:119) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.handler.ssl.SslHandler.wrap(SslHandler.java:683) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.handler.ssl.SslHandler.wrap(SslHandler.java:522) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:490) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:115) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:760) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:787) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:779) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.access$1500(AbstractChannelHandlerContext.java:39) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:1148) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:1089) [browsermob-dist-2.1.4.jar:?]
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) [browsermob-dist-2.1.4.jar:?]
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454) [browsermob-dist-2.1.4.jar:?]
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) [browsermob-dist-2.1.4.jar:?]
    at java.lang.Thread.run(Thread.java:844) [?:?]

[ERROR 2018-11-07T20:59:39,846 org.littleshoot.proxy.impl.ProxyToServerConnection] (LittleProxy-0-ProxyToServerWorker-2) (AWAITING_INITIAL) [id: 0x4ed418ee, L:/10.0.0.207:61134 - R:www.: Caught an exception on ProxyToServerConnection io.netty.handler.codec.DecoderException: java.lang.IllegalArgumentException: initialCapacity: -587 (expectd: 0+)
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:442) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488) [browsermob-dist-2.1.4.jar:?]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450) [browsermob-dist-2.1.4.jar:?]
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) [browsermob-dist-2.1.4.jar:?]
    at java.lang.Thread.run(Thread.java:844) [?:?]
Caused by: java.lang.IllegalArgumentException: initialCapacity: -587 (expectd: 0+)
    at io.netty.buffer.AbstractByteBufAllocator.validate(AbstractByteBufAllocator.java:220) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:170) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:107) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.handler.ssl.SslHandler.allocate(SslHandler.java:1472) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:991) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:902) ~[browsermob-dist-2.1.4.jar:?]
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411) ~[browsermob-dist-2.1.4.jar:?]
    ... 15 more

[INFO  2018-11-07T21:01:44,051 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (LittleProxy-JVM-shutdown-hook) Shutting down proxy server immediately (non-graceful) 
[INFO  2018-11-07T21:01:44,053 org.littleshoot.proxy.impl.DefaultHttpProxyServer] (LittleProxy-JVM-shutdown-hook) Closing all channels (non-graceful)
danishS commented 6 years ago

Can I please get some help with this pesky issue?

viktorRot commented 1 year ago

@danishS I receive the same issue. Can you please describe what actions you performed to resolve this issue?