search

tokio-rs / tokio-uring

An io_uring backed runtime for Rust
MIT License
1.14k stars 122 forks source link

Share fds between threads. #227

Open thomasbarrett opened 1 year ago

thomasbarrett commented 1 year ago

Even though tokio-uring is strictly a single-threaded runtime, I have an application that requires being able to share fds between different threads (each with their own instance of tokio-uring). Temporarily, I have been taking the approach of just leaking the tokio_uring::File struct with std::mem::forget to prevent the fd from being closed when it goes out of scope. Ideally, there would be some way to have tokio-uring "forget" a file descriptor. I was thinking that I would just submit a PR to add the IntoRawFd method for tokio_uring::File, but the logic around closing a File looks more complicated than I expected, so I figured that I would post here to see if there is a better way first.

thomasbarrett commented 1 year ago

Also, I am a little confused by the safety of the SharedFd close sequence.

  1. If there ARE in flight operations while close is called, the SharedFd will never move from a Waiting state to a Closing state. Does this mean that File::close would hang forever in this case?
  2. Even if there are no operations in-flight when submitting a close operation to the uring, is anything stopped a library user from submitting an operation to the File while it is in a "closing" state?
thomasbarrett commented 1 year ago

All right. It looks like the "consuming" signature on File::close(self) ensures that there are(1. no in-flight operations) and (2. no new operations can be started) while close is being called. This makes sense to me now.

thomasbarrett commented 1 year ago

Introduced this PR to add desired behavior.

FrankReh commented 1 year ago

I saw your PR before this issue. So my comments in that PR could be read first to avoid further confusion, but whatever.

There is a PR just submitted to io-uring that would allow coping a file descriptor from one uring to another. But ... the tokio-uring has no mechanism in place at the moment for registering an operation on the receiving uring side to know anything about the operation.

But given the kind of logic you are already using to share information between threads, maybe you don't have to care. You can use similar logic to know about the fixed slot that was picked on the receiving side.

But that's using fixed slots which may take time to bubble into tokio-uring, I don't know. And it may not fit with your needs anyway.