tokio-rs / io-uring

The `io_uring` library for Rust
Apache License 2.0
1.16k stars 129 forks source link

Handling `EAGAIN`/`EWOULDBLOCK` #262

Open jamwaffles opened 6 months ago

jamwaffles commented 6 months ago

I'm running into issues when handling multiple read CQEs simultaneously with for recv in ring.completion() { ... } where the first CQE result() is EWOULDBLOCK (or EAGAIN as they're the same code).

In the io_uring crate is there a way to either

a. Peek for ready CQEs a la io_uring_peek_cqe so I can try to read the CQE in a future iteration or b. Some configuration flags I can pass so I can wait for the CQEs to be ready with minimal latency?

As I understand it, once a CQE is ready from the ring, it cannot be re-read in a future iteration as the ring pointer is advanced in pop(). Therefore I think the answer to (a.) above is "currently, no". What about option (b.)?

As a secret third option I'd also be willing to tackle some kind of peek functionality myself but I'd need some guidance as all the docs I can find elsewhere use liburing which I don't believe this io_uring impl uses.

jamwaffles commented 6 months ago

I think I fixed this by requeuing the SQE. A snippet from my much longer and much uglier code:

for recv in unsafe { ring.completion_shared() } {
    let key = recv.user_data();

    if recv.result() == -libc::EWOULDBLOCK {
        // Get RX entry we submitted earlier and resubmit it
        let (rx_entry, _buf) = bufs.get(key as usize).expect("Could not get retry entry");

        while unsafe { ring.submission_shared().push(&rx_entry).is_err() } {
            // If the submission queue is full, flush it to the kernel
            ring.submit().expect("Internal error, failed to submit ops");
        }
    }
}

This feels pretty nasty but it steps around not being able to peek the next CQE. Would love a better way to do it though.