apache / pekko-http

The Streaming-first HTTP server/module of Apache Pekko
https://pekko.apache.org/
Apache License 2.0
150 stars 36 forks source link

"Pool shutdown unexpectedly" on client connection pool #587

Closed raboof closed 3 weeks ago

raboof commented 3 weeks ago

Getting a Pool shutdown unexpectedly error when gracefully stopping the actor system. We should probably document what can trigger this in https://pekko.apache.org/docs/pekko-http/current/troubleshooting ?

[my-system-pekko.actor.default-dispatcher-23] INFO org.apache.pekko.actor.CoordinatedShutdown - Running CoordinatedShutdown with reason [ActorSystemTerminateReason]
Aug 19 15:05:14 server generate-stats-start[1196329]: [ERROR] [08/19/2024 15:05:14.645] [my-system-pekko.actor.default-dispatcher-6] [pekko://my-system/system/pool-master] connection pool for Pool(shared->https://purchase.izettle.com:443) has shut down unexpectedly
Aug 19 15:05:14 server generate-stats-start[1196329]: java.lang.IllegalStateException: Pool shutdown unexpectedly
Aug 19 15:05:14 server generate-stats-start[1196329]:         at org.apache.pekko.http.impl.engine.client.PoolInterface$Logic.postStop(PoolInterface.scala:224)
Aug 19 15:05:14 server generate-stats-start[1196329]:         at org.apache.pekko.stream.impl.fusing.GraphInterpreter.finalizeStage(GraphInterpreter.scala:614)

AFAICT I've consumed all response entities.

(the code is at https://codeberg.org/eicas/admin-sync/src/branch/main/Zettle.scala - warning, quick hacks ahead :) )

jtjeferreira commented 3 weeks ago

AFAICT I've consumed all response entities.

In this code https://codeberg.org/eicas/admin-sync/src/branch/main/Zettle.scala#L62-L63

if (r.status != StatusCodes.OK)
  throw new IllegalStateException(s"Zettle returned status ${r.status}")

you are not consuming the response, if the status is not OK...

raboof commented 3 weeks ago

while true (sorry for the mess, not a great example :laughing: ), in that case I'd expect the script to abort with that IllegalStateException, which I don't see.

Adding debug logging, I do see the requests getting completely consumed

After upgrading both Pekko and Pekko HTTP to 1.1.0-M1, however, I can't reproduce the issue anymore. Let's close for now.