newAsyncHttpServer + waitFor server.serve() raises Exception "Connection timed out"

[aboisvert]

I'm using AsyncHttpServer (Nim 0.18.0) together with Rosencrantz (0.3.2).

My main looks like:

let server = newAsyncHttpServer()
waitFor server.serve(Port(8080), handler)

where handler is a composition of GET + POST Rosencrantz routes.

About once a day, my server will die with a "Connection timed out" error message:

server.serve() exception ref 0xb2f9fac8 --> [errorCode = 0,
parent = nil,
name = 0x12d0b4"Exception",
msg = 0xb4518030"Connection timed out\10"
"Async traceback:\10"
"  thermopi.nim(14)        thermopi\10"
"  asyncdispatch.nim(1507) waitFor\10"
"  asyncdispatch.nim(1511) poll\10"
"    ## Processes asynchronous completion events\10"
"  asyncdispatch.nim(1277) runOnce\10"
"  asyncdispatch.nim(183)  processPendingCallbacks\10"
"    ## Executes pending callbacks\10"
"  asyncmacro.nim(34)      recvLineInto_continue\10"
"    ## Resumes an async procedure\10"
"  asyncnet.nim(303)       recvLineIntoIter\10"
"  asyncfutures.nim(304)   read\10"
"  #[\10"
"    thermopi.nim(14)        thermopi\10"
"    asyncdispatch.nim(1507) waitFor\10"
"    asyncdispatch.nim(1511) poll\10"
"      ## Processes asynchronous completion events\10"
"    asyncdispatch.nim(1277) runOnce\10"
"    asyncdispatch.nim(183)  processPendingCallbacks\10"
"      ## Executes pending callbacks\10"
"    asyncmacro.nim(34)      processRequest_continue\10"
"      ## Resumes an async procedure\10"
"    asyncmacro.nim(0)       processRequestIter\10"
"    asyncfutures.nim(304)   read\10"
"  ]#\10"
"  #[\10"
"    thermopi.nim(14)        thermopi\10"
"    asyncdispatch.nim(1507) waitFor\10"
"    asyncdispatch.nim(1511) poll\10"
"      ## Processes asynchronous completion events\10"
"    asyncdispatch.nim(1277) runOnce\10"
"    asyncdispatch.nim(183)  processPendingCallbacks\10"
"      ## Executes pending callbacks\10"
"    asyncmacro.nim(34)      processClient_continue\10"
"      ## Resumes an async procedure\10"
"    asyncmacro.nim(0)       processClientIter\10"
"    asyncfutures.nim(304)   read\10"
"  ]#\10"

I've tried catching this exception and calling waitFor server.serve(Port(8080), handler) again however, I'll get various other issues doing so, such as "Address already in use", up to eventually running out of file descriptors.

It does not appear that I can effectively 1) keep my server running for long and 2) resume serving requests after a failure other than completely terminating the process, which is less than ideal because I lose some state and have some downtime between failure and recovery.

It seems the AsyncHttpServer should be resilient to these "Connection timed out" conditions and not raise an exception to begin with.

[aboisvert]

Something occurred to me after filing this issue: Should I be using runForever() instead of waitFor server.serve() ? That seems to be the common patterns I see, e.g., for Jester examples.

[ghost]

@aboisvert with runForever() async event loop will run FOREVER, and with waitFor server.serve() event loop will run until serve() returns (or throws an error)

[andreaferretti]

In theory they should be the same, as server.serve should go on indefinitely, isn't it? Here the problems seems to be that the server does not survive bad clients. I would try putting a reverse proxy such as Nginx in front of it, it may help with connection problems from the clients

[dom96]

asyncmacro.nim(0) processRequestIter

Would be nice to fix this in our stack traces.

But in any case, the exception is raised either on line 159 or line 205. So yeah, it seems like the exception just needs to be handled.