nix 0.27 `Epoll` API should be marked `unsafe`

[dimbleby]

The new Epoll API (as at nix 0.27) should be marked unsafe, probably on Epoll::add().

Currently the API has no unsafe on it, uses types from the io-safety proposal - but is in fact unsafe. Because one can add a file descriptor to an Epoll's set of interests, and then drop that file descriptor - which is just what io-safety was trying to prevent.

Useful discussion starts at https://github.com/nix-rust/nix/issues/2115#issuecomment-1695592450.

(Original report preserved below)

---

I'm interacting with a C library so I have a RawFd all along.

Previously epoll_ctl() etc took a RawFd so that was very convenient. But the new API takes Fd: AsFd which is not so convenient: I have to unsafely borrow my RawFd; and then nix is only going to .as_raw_fd() it again anyway. It's an awkward and unnecessary dance.

Fd: AsRawFd might make more sense, I reckon that's what is actually needed?

1882, cc @JonathanWoollett-Light

[JonathanWoollett-Light]

There is an explanation behind the purpose of OwnedFd and BorrowedFd here https://rust-lang.github.io/rfcs/3128-io-safety.html.

Nix is moving to this approach, it is unfortunately slow however which can introduce inconsistency that makes it a bit awkward.

[dimbleby]

hmm, it's all a bit awkward.

cf https://github.com/smol-rs/polling/pull/123 in which polling seems to have gone with a mixture of:

• use RawFd in some places, but acknowledge that it's unsafe
• use BorrowedFd in other places

I think that's probably more principled. Unless I misunderstand, this crate is currently exposing a purportedly safe API - but callers still can add file descriptors to an Epoll and close and reuse those file descriptors while the Epoll retains an interest. Which is pretty much what the io_safety stuff is trying to avoid, right?

Is that true?

[JonathanWoollett-Light]

Due to limitations in Rust's compile time evaluation I do not beleive it is currently possible to implement Epoll in this form fully safely (this would be possible with something like Zig's comptime).

It is true you can add a file descriptor with Epoll::Add then close it before closing the epoll or removing it from the interest list. You are correct I agree, this is unsafe, this should be better documented.

I am not sure what code changes would make sense here to better illustrate this.

[dimbleby]

I don't think it's only a documentation thing: it's unsafe but the API has no unsafe on it.

The suggestion - via polling - is that Epoll::add() be unsafe and accept a RawFd.

That would be the place to put the documentation saying that callers are responsible for making sure that they Epoll::delete() the file descriptor before dropping it.

[JonathanWoollett-Light]

The suggestion - via polling - is that Epoll::add() be unsafe and accept a RawFd.

I would suggest using AsRawFd so its more ergonomic, this way you could still call it with a reference to an OwnedFd.

I do not disagree with adding unsafe to it.

That would be the place to put the documentation saying that callers are responsible for making sure that the file descriptor stays alive up until the point that they Epoll::delete() it.

I do not think this actually matters. If you attempt to de-register a file descriptor that isn't registered it is safe. It will error, but it is safe.

The safety concern can be written

You must gurantee that the file descriptor given to Epoll::add is de-registered with Epoll::del or the Epoll is dropped before this file descriptor is dropped.

[dimbleby]

cool, I think we're agreeing.

I don't intend to submit an MR, but I'll update the title and thread-opener to clarify where we've got to.

[JonathanWoollett-Light]

With your update

That would be the place to put the documentation saying that callers are responsible for making sure that they Epoll::delete() the file descriptor before dropping it.

They do not need to call Epoll::delete if they just drop the epoll itself.

[asomers]

I think that Epoll::add can still be safe, as long as the Epoll captures the lifetime of the file descriptors that it's adding. There might still be use-cases where the Epoll must be long-lived even though the files it polls aren't. For those cases, we'll have to bypass I/O safety. I suggest an unsafe add_raw method. Finally, as discussed in #2134, no method should be taking an AsFd except by reference. So it would be something like this:

struct Epoll<'fd> {
    /// The actual epoll file descriptor
    pollfd: OwnedFd,
    PhantomData<'fd>
}
impl<'fd> Epoll {
    pub fn add<Fd: AsFd + 'fd>(&self, fd: &Fd) -> Result<()> {...}
    pub unsafe fn add_raw(&self, fd: RawFd) -> Result<()> {...}
}

[dimbleby]

I think that Epoll::add can still be safe, as long as the Epoll captures the lifetime of the file descriptors that it's adding.

I reckon that just doesn't match how Epoll is usually used.

Certainly in my code - but, I claim, this is the typical and intended use - the pattern is a long-lived Epoll and lots of file descriptors coming and going over the course of a program's runtime. ie I think you'll find that your add_raw() is almost always the API that people want: and in that case the lifetime-aware add() isn't worth the trouble.

The corresponding analysis in polling starts at https://github.com/smol-rs/polling/issues/38#issuecomment-1592230728, it might be interesting to compare notes.

[asomers]

There are definitely APIs which can be used with I/O Safety, but in their most compelling use cases require the BorrowedFds to be 'static. I've run into that myself, with the POSIX AIO stuff. In cases like this, I'm not sure what the best option is:

• Ignore I/O safety and operate on raw FDs only.
• Ignore I/O safety and operate on raw FDs only, but add I/O safety via some other mechanism at a higher level in some other crate.
• Add I/O safe methods of limited usefulness, plus unsafe methods that take raw file descriptors.
• Add I/O safety by using runtime checks, for example a bunch of Arc<OwnedFd>. This is my least favorite option.

[SteveLauC]

2232 will probably fix this issue

[dimbleby]

... but does it also make Epoll unusable in this typical usage https://github.com/nix-rust/nix/issues/2115#issuecomment-1741357262?

[SteveLauC]

... but does it also make Epoll unusable in this typical usage #2115 (comment)?

Well, I guess so.

And unfortunately, even though you remove a fd from the Epoll with .remove(), the borrow checker would consider this as anothe borrow:

use std::fs::File;
use std::marker::PhantomData;
use std::os::fd::{AsFd, BorrowedFd};

struct FdSet<'a> {
    _marker: PhantomData<*mut &'a ()>,
}

impl<'a> FdSet<'a> {
    fn new() -> Self {
        Self {
            _marker: PhantomData
        }
    }
    fn add(&self, fd: BorrowedFd<'a>) {}

    fn remove(&self, fd: BorrowedFd<'a>) {}
}

fn main() {
    let set = FdSet::new();
    let file = File::open("Cargo.toml").unwrap();
    set.add(file.as_fd());
    set.remove(file.as_fd());

    // It should be safe to drop the file here
    drop(file);
    drop(set);
}