SocketException: Socket closed when :as != :string

[S11001001]

On 85ce13fe913ae2f3cfc4231b835aed1303806b7e, unreliably (about 25% of the time), when using :as :identity, I get a prematurely closed socket underlying the .getContent return value.

To reproduce: GET some content with :as :identity, then (-> response :entity .getContent duck/slurp*) a few times.

I worked around it by making "getConnectionManager shutdown" happen only when as is :string.

[rnewman]

Yes, this is a thorny issue arising because HttpClient doesn't know when we're done with a resource. The only reasonable approach was to eagerly shut things down, which is slightly less efficient and — in this case, apparently — unsafe.

There is probably a race between your code using the Entity and the connection manager closing the stream. I can't reproduce it, but that doesn't mean it doesn't exist.

However, if we don't shut down the connection manager, and don't empty the stream, then the resources are never freed.

Your workaround is unfortunately not quite the correct fix, which would be to use some kind of resourcing system to keep track of the scope of the entity; when it leaves scope it would be freed.

I will consider whether there is a good solution to this problem.

In the meantime, I recommend defining your own entity-as method:

(defmethod http/entity-as :my-slurp [entity as status]
  (-> entity .getContent duck/slurp*))

then accessing the slurped result as (:entity response). This will ensure that your code runs while the socket is still open. :identity is really not ideal when compared to an entity-as method.