Idea: Custom Rust allocator for persistent transactional storage

[vvoznesensky]

I'm trying to figure out the possibility to realize an analog of POST++ for Rust. This would allow to store plain structs, containers and other instances of Rust in a transaction-protected storage over mmaped file.

Such a feature would allow, for example, to make a GUI transparent in-memory cache that mirrors server database without special local DB handling.

May I ask the reader to estimate the possibility of such machinery in Rust? I'm very new in it, my current language is C++.

[Amanieu]

The allocator API is not the right place for this. You would need a completely different representation of pointers using relative offsets, such as those used in rkyv.

[vvoznesensky]

The allocator API is not the right place for this. You would need a completely different representation of pointers using relative offsets, such as those used in rkyv.

Well, POST++ have a special flag storage::fixed to prevent a file to be mapped into different starting address for the case when references adjustment is not possible, STL containers for example. It's a limitation, but it does not mean that such approach could not be used with original pointers. Cache is usually not so precious.

Do you have another arguments against this?

Thank you.

[vvoznesensky]

The allocator API is not the right place for this. You would need a completely different representation of pointers using relative offsets, such as those used in rkyv.

Hello again.

I have realized my idea in https://crates.io/crates/tmfalloc . Please, have a look. Sorry for so many unsafe code: libc and address arithmetic as well as no Rust experience have left me no chances to avoid it.

I use your precious intrusive-collections RBTree with secret UnsafeRef fixture to manage free blocks in tmfalloc.