Open DecoyFish opened 2 years ago
Having not watched the repo is embarrassing...
Allocating a Buffer
and having it be able to dereference itself is convenient. But while this works well for a pointer or a stack buffer, what it doesn't work for is using e.g. usize
as a buffer handle; this requires asking the storage to resolve handles. Consider an allocator backed by Vec<maxalign_t>
; allocation which requires the backing store to grow means moving the existing allocations.
My current draft of the a storage API: https://cad97.github.io/storages-api/storage_api/index.html
Ultimately I've landed on fully making the API typeless and just handing out &[MaybeUninit<u8>]
. The API being complex was the biggest concern when I brought up the API for cursory discussion in the rust-lang Zulip; removing the types serves to remove a lot of incidental complexity from the core API goal: managing memory.
And I think it's still useful to have a separate trait for allocation than storage. Allocation is still a fundamental building block, and much easier to implement than a storage. Additionally, it means dynamic allocator support can be AllocStorage<dyn Allocator>
rather than dyn Storage
.
But perhaps I should look into if Allocator
can be a refinement of Storage
where Handle
is self-dereferencable... With specialization and default impl
it might even be as simple to implement.
@DecoyFish
Locally, I've prototyped a rough Box and Vec implementation along with several Allocator impls to gain a base level of assurance.
Both of those have contiguous single allocations, I would encourage you to also try to prototype something like a linked-list to ensure that your allocation/buffer API:
(I expect your API would work well, but there may always be a tiny detail...)
@CAD97
Ultimately I've landed on fully making the API typeless and just handing out
&[MaybeUninit<u8>]
.
And this was a great step forward, as it massively reduces the number of traits, removes the need for GATs, etc...
And I think it's still useful to have a separate trait for allocation than storage. Allocation is still a fundamental building block, and much easier to implement than a storage
This I am unclear on. Since Storage is more flexible, it seems like you'd want any collection to be expressed in terms of Storage for maximum flexibility for the user.
What would be useful if a user could implement an Allocator and pass it as a Storage, though requiring a small adapter like AllocStorage<A>
is not a huge hurdle.
@matthieu-m
And I think it's still useful to have a separate trait for allocation than storage.
This I am unclear on.
The key thing which convinces me that this is the fact that allocations are treated as pure. There's two places we can place the magic transformation from normal AM semantics to laundering: on the #[global_allocator]
border, or on the Allocator
border (c.f. https://github.com/rust-lang/wg-allocators/issues/101).
A really nice property of Storage
to me is that it's a pure library concept. It would significantly diminish it if the API was made magical and could omit calls, whereas that's actually desirable over the allocator boundary.
Having two separate traits allows us to separate the two.
...
Though I have another interesting design that might be convincing me otherwise...
But I think I have a conclusion from this experiment:
SharedStorage
design as an alternative to the Multiple
/SharedMutability
design
MultipleStorage
s not providing &self
allocation or SharedMutability
is rare in practice&mut
and &
versionsAllocator
is a well defined refinement of Storage
, even ignoring pinning
I mentioned this in the latest ArrayVec RFC: https://github.com/rust-lang/rfcs/pull/3316#issuecomment-1247587175
Here's another take on enabling a storage-backed Allocator trait with minimal changes to the current implementation. This is inspired by @matthieu-m's awesome storage-poc (see also #79).
Locally, I've prototyped a rough Box and Vec implementation along with several Allocator impls to gain a base level of assurance. However, I'm somewhat new to both Rust and this feature so I might have overlooked key issues. Please also let me know if I have or if this (or a similar) proposal has already been considered and rejected/deferred.
Objectives
Allocator
trait object safe (this was only partially successful -Allocator<Buffer = NonNull<u8>>
is object safe, which appears to still meet the spirit of the request in #83)Issues
Currently it faces the same limitation as @matthieu-m's storage-poc: the Box implementation is not coercible because a type can't effectively implement
CoerceUnsized
while containing<T as Pointee>::Metadata
. This issue is being discussed in rust-lang's #81513.While not blocking, it would reduce the required constraints in data structure implementations if
Pointee::Metadata
requiredMetadata<Self>
.Proposal
PS: I am not good at naming. My dog's fairly lucky that he didn't end up named "Dog". Please let me know if you have any suggestions regarding better names.