Open mpilquist opened 6 months ago
In such cases, the underlying selector key has been marked interested in reads but the selector never indicates the socket is ready, and manually inspecting the key confirms readyOps = 0.
What about actually attempting to read from the socket? i.e. is it possible there is something to read on the socket even though the selector is not becoming read-ready?
Yeah, our implementation attempts the read here: https://github.com/typelevel/fs2/blob/52b9e5d37fde447705ddf3c05cd15934e1eac0a6/io/jvm/src/main/scala/fs2/io/net/AsynchronousDatagramSocketGroup.scala#L183 -- read1
calls channel.receive
. No luck.
I was looking in to the intermittent test failures in #3376 and seemed to have found a bug in our support for UDP on the JVM. Consider the following application, which starts a UDP echo server and then loops forever, creating a client socket, sending a datagram to the server, and waiting for a response.
If a response hasn't been received within a second, the read is cancelled and the message is resent. Retries continue until a datagram has been read successfully from the client socket.
The program keeps a running tally of the number of attempts it took to receive the echoed response.
Running this results in output like:
In this example, there were 91,421 client sockets that received a response to their first attempt, 1 socket that required a single retry, and 1 socket that required 13 retries.
When this program is run on the JVM (via
scala-cli run udp.scala
), these retries occur more frequently than expected due to UDP packet loss. More worrisome, sometimes the retries continue forever. In such cases, the underlying selector key has been marked interested in reads but the selector never indicates the socket is ready, and manually inspecting the key confirms readyOps = 0.Some sample outputs:
Stalled example:
Non-stalled example but with extensive retries:
Adding a small delay after opening the client socket and before writing to it decreases the likelihood of a stall happening, which seems to point towards some type of race condition at socket creation / registration time.
Running the same program on Scala.js (via
scala-cli run --js --js-module-kind es udp.scala
) results in no drops & no stalls.