compio-rs / compio

A thread-per-core Rust runtime with IOCP/io_uring/polling.
MIT License
420 stars 37 forks source link

Handle EAGAIN on IO uring submit #51

Closed DXist closed 1 year ago

DXist commented 1 year ago

io_uring_enter could return EAGAIN on temporal resource shortage.

Driver could handle it and retry submission.

EAGAIN The kernel was unable to allocate memory for the request, or otherwise ran out of resources to handle it. The application should wait for some completions and try again.

Offtopic: I guess ENXIO cannot happen because driver doesn't drop IoUring instance.

Berrysoft commented 1 year ago

Sounds that we can simply treat it the same as EBUSY.

DXist commented 1 year ago

Yes. I've looked into Tigerbeetle's code and they treat these errors in the same way.

We should also handle EINTR in case user installs own signal handler and the same thread catches it.

Berrysoft commented 1 year ago

EINTR has been handled in the runtime.

DXist commented 1 year ago

Handling EINTR in the driver could provide convenience for external runtimes.

On the other hand different callers could have different requirements for wait timeout. For example, if a caller expects waiting till absolute timeout moment it have to recalculate new relative timeout for retry.

Maybe it's possible to support cross-platform absolute timeout operation?

For IO uring with absolute Timeout operation persisted in the submission queue it's possible just to retry submission in the driver side without recalculation of relative timeout.

Berrysoft commented 1 year ago

The timeout is never absolute and impossible to be absolute. There are many reasons of returning early:

Yes, the runtime need to recalculate a new relative timeout. It is the duty of the runtime. Every runtime does it.

DXist commented 1 year ago

I'm interested in just EINTR case. Absolute timeout will increase precision of idle wait if driver client and kernel use the same clock source. Handling completes a bit past to the absolute timeout is ok for me.

Berrysoft commented 1 year ago

If you want it, open a PR for it, and make sure you do it right for all three drivers.