Closed JackKelly closed 5 months ago
You may be wondering: Why didn't I implement from_raw_parts
for BytesMut
? My understanding (which could be wrong) is that BytesMut
uses Vec<u8>
internally. Vec
will re-allocate if users push more elements into the Vec
than will fit into the current Vec::capacity
. And, AFAIK, there's no easy way to safely re-allocate a Vec
whilst maintaining alignment.
Also, crucially, for my use-case Bytes
is perfectly sufficient.
I'm sorry, but due to the destructor using a different alignment, this is not sound.
Something like this will most likely require the proposed vtable api, but the bytes crate does not have enough maintainer bandwidth to review a new vtable API right now.
OK, no worries, thanks for the very quick review!
I'd like to keep track of progress towards a vtable API in Bytes
(although I appreciate that it might take a while!). Is this the best issue to subscribe to:
Yes, that issue is fine.
Use-case
I need to construct buffers which are aligned to 512-byte boundaries, in order to read data from disk using
O_DIRECT
on Linux (see issue #600 for a user who has the same use-case as me).Once the operating system has written into these buffers, I'd love to wrap the buffer in
Bytes
so I can:object_store
's API which returnsBytes
.Wrapping an aligned buffer is possible today using
Bytes
as it current stands (although I'm not entirely certain if this will be safelydrop
'd in all situations, becauseBytes::drop
doesn't know the alignment??):However, that requires two unnecessary steps:
slice::from_raw_parts_mut
andBox::from_raw
.This PR adds a very simple
Bytes::from_raw_parts
method (which is mostly copied fromimpl From<Box[u8]> for Bytes
).With this new PR, wrapping an aligned buffer in
Bytes
is a little more ergonomic for users, and a little more computationally efficient:(This is my first PR to a Rust open-source project, so please LMK if I've done anything wrong!)
TODO:
static Vtable
whereVtable.drop
knows the alignment of the data to drop. Is that correct? Is that even possible, given thatstruct Vtable.drop
can't accept a closure?! Maybe the solution is to create a macro to insert the alignment into the drop function (although that will only work when we know the alignment at compile time... although, for aligned buffers forO_DIRECT
, we could create a handful of alignments (e.g. 512-bytes and 4096-bytes) at compile-time, and select the correct one at runtime.)impl From<Box[u8]> for Bytes
by callingBytes::from_raw_parts
.Related
600
601
607