Closed carbotaniuman closed 1 year ago
We talked about this in today's libs-api meeting.
BorrowedBuf
is specifically a type for doing I/O. It tracks more than you might want for potential other applications, and we don't want to simultaneously try to be the most optimal type for I/O and also be a general type meant for other uses.
For use with i8
or c_char
, that's a zero-cost safe conversion; you can use a BorrowedBuf
that operates on u8
and then treat the result as i8
or c_char
.
For use with other types, we're not closing the door on the future possibility of either adding a separate type or making BorrowedBuf an alias for that type, but we don't want to do that at this time.
Proposal
Problem statement
Rust does not provide great provisions for reading data into uninitialized buffers, and while
BorrowedBuf
andBorrowedCursor
solve the problem foru8
in the context of I/O, generifying the struct would solve this problem for the entire ecosystem, instead of just for I/O.Motivating examples or use cases
My main use case is for safe FFI wrappers, where there are a lot of libraries that take a pointer and a length that could be massaged to work with
BorrowedBuf
, either as a caller or for reimplementation in safe(r) Rust. Some examples include I/O that deals inc_char
or evenwchar
due to legacy APIs.https://docs.rs/rustix/latest/rustix/fs/fn.flistxattr.html
In addition, the general idea behind reading data into an uninitialized buffer is general enough that existing Rust code is bound to run into the issue, such as rayon.
https://github.com/rust-lang/rust/issues/78485#issuecomment-828561911
Solution sketch
The solution is to add a (potentially defaulted to
u8
) generic parameter after the existing lifetime parameters. The documentation of the struct may have to change to reflect it dealing in elements and not in bytes.Alternatives
Another solution could be a
BorrowedBuf
being a typealias to a new struct that is generic, or creating an entirely new struct for the purpose of storing arbitrary items, as opposed to I/O. Creating a new struct very redundant in my opinion, and having a single way to talk about reading into uninitialized memory seems preferable.Links and related work
This was discussed during the RFC and rejected as future work. I think that generifying the API to allow for progressive initialization would be useful for other many use cases, even in the presence of a separate
Vec
-like data structure that works off of uninitialized memory.https://github.com/rust-lang/rfcs/pull/2930#discussion_r426967877
What happens now?
This issue is part of the libs-api team API change proposal process. Once this issue is filed the libs-api team will review open proposals as capability becomes available. Current response times do not have a clear estimate, but may be up to several months.
Possible responses
The libs team may respond in various different ways. First, the team will consider the problem (this doesn't require any concrete solution or alternatives to have been proposed):
Second, if there's a concrete solution: