search

browserup / browserup-proxy

BrowserUp Proxy is a free utility to watch, test, and manipulate web application network traffic and performance.
https://browserup.com
Apache License 2.0
164 stars 41 forks source link

NPE when failing to decode response from server #288

Open natalliabarouka opened 4 years ago

natalliabarouka commented 4 years ago

Hi! We are seeing NPE while BrowserUp sessions while capturing data from https://smartystreets.com/products/apis/us-street-api within our selenium tests. Other tests are looking fine.

Steps to reproduce the behavior:

  1. Add Proxy - we use built-in Selenium mode, version 2.1.1
  2. Go to us-autocomplete.api.smartystreets.com
  3. Capture requests
  4. See error

Expected behavior No NPE errors in log. See part of log related to the issue below:

2020-06-29 16:12:05,827 DEBUG org.littleshoot.proxy.impl.ProxyConnection [LittleProxy-0-ProxyToServerWorker-0] (AWAITING_INITIAL) [id: 0x6c6e2630, L:/100.64.100.196:58625 - R:us-autocomplete.api.smartystreets.com/34.66.37.1:443]: Reading: DefaultHttpResponse(decodeResult: success, version: HTTP/1.1) HTTP/1.1 408 Request Time-out content-length: 110 cache-control: no-cache content-type: text/html connection: close 2020-06-29 16:12:05,828 DEBUG org.littleshoot.proxy.impl.ProxyToServerConnection [LittleProxy-0-ProxyToServerWorker-0] (AWAITING_INITIAL) [id: 0x6c6e2630, L:/100.64.100.196:58625 - R:us-autocomplete.api.smartystreets.com/34.66.37.1:443]: Received raw response: DefaultHttpResponse(decodeResult: success, version: HTTP/1.1) HTTP/1.1 408 Request Time-out content-length: 110 cache-control: no-cache content-type: text/html connection: close 2020-06-29 16:12:05,828 DEBUG org.littleshoot.proxy.impl.ProxyToServerConnection [LittleProxy-0-ProxyToServerWorker-0] (AWAITING_INITIAL) [id: 0x6c6e2630, L:/100.64.100.196:58625 - R:us-autocomplete.api.smartystreets.com/34.66.37.1:443]: Remembering the current response. 2020-06-29 16:12:05,828 ERROR org.littleshoot.proxy.impl.ProxyToServerConnection [LittleProxy-0-ProxyToServerWorker-0] (AWAITING_INITIAL) [id: 0x6c6e2630, L:/100.64.100.196:58625 - R:us-autocomplete.api.smartystreets.com/34.66.37.1:443]: Caught an exception on ProxyToServerConnection java.lang.NullPointerException: null at org.littleshoot.proxy.impl.ProxyUtils.isHEAD(ProxyUtils.java:321) at org.littleshoot.proxy.impl.ClientToProxyConnection.respond(ClientToProxyConnection.java:430) at org.littleshoot.proxy.impl.ProxyToServerConnection.respondWith(ProxyToServerConnection.java:546) at org.littleshoot.proxy.impl.ProxyToServerConnection.readHTTPInitial(ProxyToServerConnection.java:259) at org.littleshoot.proxy.impl.ProxyToServerConnection.readHTTPInitial(ProxyToServerConnection.java:86) at org.littleshoot.proxy.impl.ProxyConnection.readHTTP(ProxyConnection.java:144) at org.littleshoot.proxy.impl.ProxyConnection.read(ProxyConnection.java:122) at org.littleshoot.proxy.impl.ProxyToServerConnection.read(ProxyToServerConnection.java:229) at org.littleshoot.proxy.impl.ProxyConnection.channelRead0(ProxyConnection.java:582) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at org.littleshoot.proxy.impl.ProxyConnection$ResponseReadMonitor.channelRead(ProxyConnection.java:732) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:316) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:303) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:417) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:271) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at org.littleshoot.proxy.impl.ProxyConnection$BytesReadMonitor.channelRead(ProxyConnection.java:686) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355) at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1470) at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1219) at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1266) at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:493) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:432) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:271) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at java.base/java.lang.Thread.run(Thread.java:832) 2020-06-29 16:12:05,829 INFO org.littleshoot.proxy.impl.ProxyToServerConnection [LittleProxy-0-ProxyToServerWorker-0] (AWAITING_INITIAL) [id: 0x6c6e2630, L:/100.64.100.196:58625 - R:us-autocomplete.api.smartystreets.com/34.66.37.1:443]: Disconnecting open connection to server 2020-06-29 16:12:05,829 DEBUG org.littleshoot.proxy.impl.ProxyConnection [LittleProxy-0-ProxyToServerWorker-0] (AWAITING_INITIAL) [id: 0x6c6e2630, L:/100.64.100.196:58625 ! R:us-autocomplete.api.smartystreets.com/34.66.37.1:443]: Reading: EmptyLastHttpContent 2020-06-29 16:12:05,829 DEBUG org.littleshoot.proxy.impl.ProxyConnection [LittleProxy-0-ProxyToServerWorker-0] (AWAITING_INITIAL) [id: 0x6c6e2630, L:/100.64.100.196:58625 ! R:us-autocomplete.api.smartystreets.com/34.66.37.1:443]: Dropping message because HTTP object was not an HttpMessage. HTTP object may be orphaned content from a short-circuited response. Message: EmptyLastHttpContent 2020-06-29 16:12:05,829 DEBUG org.littleshoot.proxy.impl.ProxyConnection [LittleProxy-0-ProxyToServerWorker-0] (AWAITING_INITIAL) [id: 0x6c6e2630, L:/100.64.100.196:58625 ! R:us-autocomplete.api.smartystreets.com/34.66.37.1:443]: Reading: DefaultFullHttpResponse(decodeResult: failure(java.lang.IllegalArgumentException: invalid version format: <HTML><BODY><H1>408), version: HTTP/1.0, content: UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeHeapByteBuf(ridx: 0, widx: 0, cap: 0)) HTTP/1.0 999 Unknown 2020-06-29 16:12:05,830 DEBUG org.littleshoot.proxy.impl.ProxyToServerConnection [LittleProxy-0-ProxyToServerWorker-0] (AWAITING_INITIAL) [id: 0x6c6e2630, L:/100.64.100.196:58625 ! R:us-autocomplete.api.smartystreets.com/34.66.37.1:443]: Received raw response: DefaultFullHttpResponse(decodeResult: failure(java.lang.IllegalArgumentException: invalid version format: <HTML><BODY><H1>408), version: HTTP/1.0, content: UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeHeapByteBuf(ridx: 0, widx: 0, cap: 0)) HTTP/1.0 999 Unknown 2020-06-29 16:12:05,830 DEBUG org.littleshoot.proxy.impl.ProxyToServerConnection [LittleProxy-0-ProxyToServerWorker-0] (AWAITING_INITIAL) [id: 0x6c6e2630, L:/100.64.100.196:58625 ! R:us-autocomplete.api.smartystreets.com/34.66.37.1:443]: Could not parse response from server. Decoder result: failure(java.lang.IllegalArgumentException: invalid version format: <HTML><BODY><H1>408) 2020-06-29 16:12:05,830 DEBUG org.littleshoot.proxy.impl.ProxyToServerConnection [LittleProxy-0-ProxyToServerWorker-0] (AWAITING_INITIAL) [id: 0x6c6e2630, L:/100.64.100.196:58625 ! R:us-autocomplete.api.smartystreets.com/34.66.37.1:443]: Remembering the current response.

I also saw similar issue handled already here, but not sure if it's the exact one https://github.com/adamfisk/LittleProxy/pull/330