EncoderException: IllegalStateException: Unexpected message type: DefaultHttpRequest, state: 1

ljwagerfield commented 4 years ago

When using an HTTP4S 0.21.1 client (aync-http) and a HTTP4S server (blaze), I frequently receive the following errors on GET requests that occur immediately after POST requests with a string body "BOOM!".

[INFO] org.http4s.blaze.channel.ServerChannel - Closing NIO1 channel /127.0.0.1:9010
io.netty.handler.codec.EncoderException: java.lang.IllegalStateException: unexpected message type: DefaultHttpRequest, state: 1
    at io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:107)
    at io.netty.channel.CombinedChannelDuplexHandler.write(CombinedChannelDuplexHandler.java:346)
    at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:715)
    at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:707)
    at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:790)
    at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:700)
    at io.netty.handler.stream.ChunkedWriteHandler.doFlush(ChunkedWriteHandler.java:332)
    at io.netty.handler.stream.ChunkedWriteHandler.flush(ChunkedWriteHandler.java:133)
    at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:748)
    at io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:763)
    at io.netty.channel.AbstractChannelHandlerContext$WriteTask.run(AbstractChannelHandlerContext.java:1089)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: java.lang.IllegalStateException: unexpected message type: DefaultHttpRequest, state: 1
    at io.netty.handler.codec.http.HttpObjectEncoder.encode(HttpObjectEncoder.java:86)
    at io.netty.handler.codec.http.HttpClientCodec$Encoder.encode(HttpClientCodec.java:167)
    at io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:89)
    ... 16 more

Reproduction code: https://github.com/ljwagerfield/http4s-request-body-error

More detail:

The problem is caused by withEntity("BOOM!"). However, the POST request with that body succeeds, but the subsequent GET fails: the GET is never sent to the server, and instead the above exception occurs.

The problem goes away if I use a different backend (e.g. python -m SimpleHTTPServer), implying Blaze Server manages to get HTTP4S AHC into a bad state.

The fact that HTTP4S AHC is able to be put into such a state by a server, such that the subsequent request fails before it's even sent to the server implies a bug with HTTP4S AHC... there may also be a separate bug with Blaze Server.

rossabaker commented 4 years ago

I have seen this error while trying to fix another error, which may be a clue what's happening here: https://github.com/AsyncHttpClient/async-http-client/issues/1660

ljwagerfield commented 4 years ago

Interesting that yours happens 'under load': mine does not.

More interestingly, I found the sequence of GET, GET, DELETE, DELETE, POST (with body), GET seemed to make the error surface more frequently than other combinations:

val requests: List[Request[Task]] = List(
  Request(
    Method.GET,
    Uri.unsafeFromString(url)
  ),
  Request(
    Method.GET,
    Uri.unsafeFromString(url)
  ),
  Request(
    Method.DELETE,
    Uri.unsafeFromString(url)
  ),
  Request(
    Method.DELETE,
    Uri.unsafeFromString(url)
  ),
  Request(
    Method.POST,
    Uri.unsafeFromString(url),
  ).withEntity(
    "BANG!"
  ),
  Request(
    Method.GET,
    Uri.unsafeFromString(url)
  ),
)

http4s / http4s-async-http-client

EncoderException: IllegalStateException: Unexpected message type: DefaultHttpRequest, state: 1 #2