search

akka / akka-edge-rs

Akka Edge support in Rust
https://doc.akka.io/docs/akka-edge/current/
Other
9 stars 1 forks source link

ERROR hyper::server::tcp] accept error: Too many open files (os error 24) #58

Closed patriknw closed 1 year ago

patriknw commented 1 year ago

A convoluted way to reproduce this:

  1. start samples/grpc/restaurant-drone-deliveries-service-scala

    sbt -Dconfig.resource=local1.conf run

  2. start samples/grpc/local-drone-control-scala

    sbt run

  3. report location

    grpcurl -d '{"drone_id":"drone1", "coordinates": {"longitude": 18.07125, "latitude": 59.41834}, "altitude": 5}' -plaintext 127.0.0.1:8080 local.drones.DroneService.ReportLocation

  4. stop local-drone-control-scala

  5. start iot-service-rs, which will connect to the "invalid" restaurant-drone-deliveries-service-scala

  6. Idle a few minutes

  7. report location again, which iot-service-rs doesn't know about

    grpcurl -d '{"drone_id":"drone1", "coordinates": {"longitude": 18.07125, "latitude": 59.41834}, "altitude": 5}' -plaintext 127.0.0.1:8080 local.drones.DroneService.ReportLocation

That results in too many open files in iot-service-rs

[2023-10-19T09:19:48.099Z ERROR hyper::server::tcp] accept error: Too many open files (os error 24)
[2023-10-19T09:19:49.102Z ERROR hyper::server::tcp] accept error: Too many open files (os error 24)
[2023-10-19T09:19:50.650Z ERROR hyper::server::tcp] accept error: Too many open files (os error 24)
[2023-10-19T09:19:53.648Z ERROR hyper::server::tcp] accept error: Too many open files (os error 24)
[2023-10-19T09:19:57.834Z ERROR hyper::server::tcp] accept error: Too many open files (os error 24)
huntc commented 1 year ago

I suspect what’s happening here is that the deserialisation of an unexpected message is causing the connection to be dropped, but then it is re-opened again before the last connection is cleaned up. I’ll throttle these such that the connections can’t be re-opened rapidly.