Closed progval closed 1 year ago
Awwwwww... that's me cleaning up the code asking Tommaso "are we using this?" and of course he says no because we aren't using it. But. Other. People. Might. 🙈
Well, one possibility is simply resurrecting the code. It should still work. Let me have a look.
BTW, I don't know if that would be OK for you, but probably it would be sufficient to implement VSlice for &[u8], since MMap implements AsRef<[u8]>.
So, from what I see you need MmapMut. Question: can't you transmute the &mut [u8] for MmapMut into a &mut[usize]? Because CompactArray is implemented for that.
Note that there was a too strong trait bound for CompactArray (Vec
Like, this works:
use std::fs::{File, OpenOptions};
use std::io::Write;
use sux::prelude::CompactArray;
use sux::prelude::*;
pub fn main() {
let mut file = File::create("test.bin").unwrap();
file.write_all(&[0_u8, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])
.unwrap();
drop(file);
let file = OpenOptions::new()
.read(true)
.write(true)
.open("test.bin")
.unwrap();
let mut mmap = unsafe {
mmap_rs::MmapOptions::new(16)
.unwrap()
.with_file(file, 0)
.map_mut()
.unwrap()
};
let mmap_usize = unsafe { std::slice::from_raw_parts_mut(mmap.as_mut_ptr() as *mut usize, 2) };
let mut c = unsafe { CompactArray::from_raw_parts(mmap_usize, 4, 32) };
for i in 0..c.len() {
println!("{}", c.get(i));
c.set(i, 5);
println!("{}", c.get(i));
}
}
You can make it healthier using bytemuck, transmute, align_to, etc.
can't you transmute the &mut [u8] for MmapMut into a &mut[usize]?
I'm afraid not, it's not guaranteed to be padded to end with a whole usize
Er... ahem... huh... I'm afraid you're not familiar with the code you used up to now 😂
impl VSlice for mmap_rs::MmapMut {
#[inline(always)]
unsafe fn get_unchecked(&self, index: usize) -> u64 {
debug_assert!(index < self.len(), "{} {}", index, self.len());
let ptr = (self.as_ptr() as *const u64).add(index);
std::ptr::read(ptr)
}
}
Ah, my bad, it is padded. I'll try your suggestions, thanks
No, I mean, I didn't generate your data so I don't know but the main() I wrote took the code exactly from the implementation you were using. Either none or both work...
I mean I checked my existing file again, and it does have padding.
As my Node2Type
struct needs to own the mmap, I'll have to call CompactArray::from_raw_parts
on every call to Node2Type::get
or Node2Type::set
. It looks like it would be "free" with the current CompactArray
code, but do you expect it to remain so in the future?
I did it by turning my Node2Type<B: VSlice>
into a pair of traits (Node2Type
and Node2TypeMut
) and a pair of structs (one for generic AsRef<[usize]>
/ AsMut<[usize]>
that I could probably generalize to VSlice; and one for Mmap
/MmapMut
), by calling CompactArray::from_raw_parts
on every access: https://gitlab.softwareheritage.org/swh/devel/swh-graph/-/blob/db07cb81f722549085f0f694bdcd204378fc4d7c/rust/src/map/node2type.rs
I didn't check the compiler actually makes it zero-cost though
But if you implement AsRef<[usize]> for MMap it should work, no? At that point the current VSlice implementation will pick it up.
Am I missing something?
Ah yes, of course. I needed to take a step back, thanks
(and I had to introduce a newtype as both AsRef and Mmap are external to my crate, but no big deal)
Hi,
In swh-graph, we use a
CompactArray<Mmap>
, which isn't supported since e468cedf68a16ffe0ef73b9703342f665dc6b9dc: https://gitlab.softwareheritage.org/swh/devel/swh-graph/-/blob/ec014b29c2361a13bbfa55f3f509be74572d7699/rust/src/map/node2type.rs#L10I tried to generalize VSlice and CompactArray to non-usize types (I need
u8
here) but I'm getting lost in satisfying the type-checker for these implementations (especially the interactions betweenVSliceCore<Type>
,VSliceCore<AtomicType>
, andVSliceAtomic<AtomicType>
:but I can't also provide blanket impls for
N: AtomicNumber
because Rust doesn't support providing blanket impls for two different traits ("some third-party struct might implement both traits").So I also tried with macros:
Any idea to resolve this?