nonblocking pipes?

[oconnor663]

This would require switching to CreateNamedPipe on Windows. Which we might want to expose anyway.

[oconnor663]

See this comment in the Rust standard library:

    // Note that we specifically do *not* use `CreatePipe` here because
    // unfortunately the anonymous pipes returned do not support overlapped
    // operations. Instead, we create a "hopefully unique" name and create a
    // named pipe which has overlapped operations enabled.
    //
    // Once we do this, we connect do it as usual via `CreateFileW`, and then
    // we return those reader/writer halves. Note that the `ours` pipe return
    // value is always the named pipe, whereas `theirs` is just the normal file.
    // This should hopefully shield us from child processes which assume their
    // stdout is a named pipe, which would indeed be odd!

[link2xt]

I have an idea to expose async Rust queue (channel) to C code for use with Qt event loop and QSocketNotifier. For that I was going to create a pipe with os_pipe in Rust code and give reader half to C code via FFI. C code will give the file descriptor to the Qt event loop for polling on it, and Rust code will write a byte to the pipe each time it puts something into the channel to wake up event processing loop. C code will then call an FFI-exposed Rust function that reads one byte from the pipe and extracts a queue item for C code to process.

But for that I need nonblocking pipes, because otherwise I can't solve the potential deadlock. Processing an event may result in calling a Rust function from C code that will try writing to the queue and pipe, potentially lock on the pipe. Nobody will read from the pipe, because event processing loop waits for the Rust code to return.

And even if this problem is somehow solved and I can avoid Rust code blocking on the pipe when it is full, it means the queue will have more items than the pipe. So C code needs to try to exhausting the queue whenever it is notified regardless of the number of notifications in the pipe, which means FFI function trying to extract an item from the queue should not block even if the pipe is empty.

If this is relevant, the idea is for https://github.com/deltachat/deltachat-core-rust/. Current C API documented at https://c.delta.chat/ requires using a separate event loop thread as shown in synopsis. I want to have an additional dc_get_event_emitter_fd() API to poll on it and use the library asynchronously without threads.

@oconnor663 Are you by chance planning to implement this? The crate looks maintained, but the issue is somewhat old. Nonblocking mode would be really useful for the use case of making event-based C bindings to async Rust code.

[oconnor663]

No, I won't be adding new features to this crate anytime soon. That's partly because I'm working on other things, and partly because I view this crate as a simple abstraction for the most common use cases. In more advanced use cases like the one you're proposing, you might need to do more advanced things like configuring pipe buffer sizes, which this crate will probably never expose. Especially since you're already writing C FFI code, it probably makes the most sense to create pipes directly with libc and win32 calls as appropriate to the platform. What do you think?

[link2xt]

I don't think I need to configure pipe size, first of all because it is not portable. As far as I know OpenBSD has no API to change pipe capacity. For purposes of event notification it is ok if the pipe occasionally overflows or underflows. When reading, I can try reading a byte from the pipe first and then read from the queue even if the pipe was empty. On writing, I will write to the queue first and then write to the pipe to notify the receiver, no problem if the pipe is full already.

All I need to support event notification use case is a way to make the pipe nonblocking.

I don't want to do it in the main crate, because it is completely free of unsafe code and platform-specific sections currently. Everything dealing with pipes will have to reside in a separate unsafe crate anyway on which the main crate will depend, so I will end up with an os_pipe clone for non-blocking pipes.

The code reading/writing to the created pipes can use standard read/write operations and check for io Error kind of WouldBlock in a portable way.

Would the code adding support for nonblocking pipes be accepted to this crate if someone (hopefully me) finds the time to code this feature?