Trying to understand PBUF_RING for Rust implementation

[SUPERCILEX]

I'm looking to implement abstractions for pbufs in Rust: https://github.com/tokio-rs/tokio-uring/issues/112

I just want to confirm my understanding:

• Registering the ring is about giving userspace and the kernel a shared ring to do atomic ops on the head/tail pointers (in the case of userspace, just the tail). It's basically a fancy list of allocations.
• The registered memory only needs space to hold the ring.
• struct io_uring_buf may contain arbitrary allocations created whenever and however the user likes. The only requirement is that ownership be relinquished to the kernel once added to the ring.
• The ring memory will not be freed by the kernel. More generally, once the io_uring is destroyed, ownership of all memory involved with the ring returns to the user whose job it is to clean up.

[isilence]

* Registering the ring is about giving userspace and the kernel a shared ring to do atomic ops on the head/tail pointers (in the case of userspace, just the tail). It's basically a fancy list of allocations.

Right. That paired with IOSQE_BUFFER_SELECT/etc. to ask the kernel to alloc from it.

* The registered memory only needs space to hold the ring.

Not sure I follow. Are you asking how much memory the kernel allocates internally for pbuf and at runtime using it? That would be roughly equal to the size of the ring. And AFAIK everything needed is allocated at pbuf registration and nothing at runtime.

* `struct io_uring_buf` may contain arbitrary allocations created whenever and however the user likes. The only requirement is that ownership be relinquished to the kernel once added to the ring.

Yes, those are ordinary users pointer, same ones you would pass to read(2)/etc. The kernel doesn't care really about the ownership, but the userspace should otherwise you may get multiple parallel IO into the same buffer effectively corrupting the data.

* The ring memory will not be freed by the kernel. More generally, once the io_uring is destroyed, ownership of all memory involved with the ring returns to the user whose job it is to clean up.

If you mean memory for provided buffers, i.e. pointers you pass in the ring, then yes.

[SUPERCILEX]

Awesome, thanks! Sounds like we're on the same page.

Not sure I follow. Are you asking how much memory the kernel allocates internally for pbuf and at runtime using it? That would be roughly equal to the size of the ring. And AFAIK everything needed is allocated at pbuf registration and nothing at runtime.

In this example https://github.com/axboe/liburing/blob/master/examples/io_uring-udp.c, you allocate the buffers and ring in the same mmap, but based on your reply that's just a convenient coincidence. Basically I meant that the memory given to the register syscall only needs space for the ring header and entries even though the user might choose to allocate extra space for adjacent buffers.

[DylanZA]

@SUPERCILEX this is correct.

the buffers that the ring points to can be anywhere

[SUPERCILEX]

Awesome, thanks!

[SUPERCILEX]

What's the rationale for the tail being a u16? On architectures like risc-v, 16-bit atomics aren't supported.