search

http4s / http4s-async-http-client

async http client implementation for http4s clients
Apache License 2.0
0 stars 2 forks source link

async-http-client: duplicate handler #1

Open rossabaker opened 5 years ago

rossabaker commented 5 years ago

Under load, sending requests with bodies with the async HTTP client backend fails intermittently:

[info] java.lang.IllegalArgumentException: Duplicate handler name: request-body-streamer
[info]  at io.netty.channel.DefaultChannelPipeline.checkDuplicateName(DefaultChannelPipeline.java:1066)
[info]  at io.netty.channel.DefaultChannelPipeline.filterName(DefaultChannelPipeline.java:284)
[info]  at io.netty.channel.DefaultChannelPipeline.addLast(DefaultChannelPipeline.java:204)
[info]  at io.netty.channel.DefaultChannelPipeline.addLast(DefaultChannelPipeline.java:195)
[info]  at org.asynchttpclient.netty.request.body.NettyReactiveStreamsBody.write(NettyReactiveStreamsBody.java:63)
[info]  at org.asynchttpclient.netty.request.NettyRequestSender.writeRequest(NettyRequestSender.java:426)
[info]  at org.asynchttpclient.netty.request.NettyRequestSender.sendRequestWithOpenChannel(NettyRequestSender.java:259)
[info]  at org.asynchttpclient.netty.request.NettyRequestSender.sendRequestWithCertainForceConnect(NettyRequestSender.java:141)
[info]  at org.asynchttpclient.netty.request.NettyRequestSender.sendRequest(NettyRequestSender.java:113)
[info]  at org.asynchttpclient.DefaultAsyncHttpClient.execute(DefaultAsyncHttpClient.java:241)
[info]  at org.asynchttpclient.DefaultAsyncHttpClient.executeRequest(DefaultAsyncHttpClient.java:210)
[info]  at org.http4s.client.asynchttpclient.AsyncHttpClient$.$anonfun$allocate$4(AsyncHttpClient.scala:48)
[info]  at org.http4s.client.asynchttpclient.AsyncHttpClient$.$anonfun$allocate$4$adapted(AsyncHttpClient.scala:47)

Similar to https://github.com/AsyncHttpClient/async-http-client/issues/1627.

rossabaker commented 5 years ago

Reproduction, based on a gist from @sherwinschiu.

rossabaker commented 5 years ago

I think the problem is that when the NettySubscriber cancels, the StreamSubscriber is under no obligation to send an onComplete or onError event. The NettySubscriber should remove the channel handler when it cancels. But cancellation is handled by the parent HandlerSubscriber from netty-reactive-streams, so there's no hook to clean up the resources.

rossabaker commented 5 years ago

Better discussion: https://github.com/AsyncHttpClient/async-http-client/issues/1660