PSeitz / lz4_flex

Fastest pure Rust implementation of LZ4 compression/decompression.
MIT License
460 stars 31 forks source link

Use Vec::extend_from_within() in place of custom unsafe code #18

Open Shnatsel opened 3 years ago

Shnatsel commented 3 years ago

Vec::extend_from_within() is a newly stabilized function that allows appending a part of a vector to itself.

Looks like it could replace some uses of unsafe code in lz4_flex, e.g. https://github.com/PSeitz/lz4_flex/blob/053483689099420c8576a378d8bf4e1d1fe2964f/src/block/decompress.rs#L15-L16

If it turns out to be not as good as the unsafe variant (I see that you're specifically avoiding memcpy), perhaps it could still make the safe variant faster.

PSeitz commented 3 years ago

I did some experiments after your comment on reddit last time. The unsafe version was faster, since there was still some overhead in comparison to the pointer approach.

In the meantime the block api changed and it uses slices now as parameters, so extend_from_slice is not applicable anymore. But there were some gains in the safe variant by using the slice copy methods.

Shnatsel commented 3 years ago

The slice-based approach as used right now is sadly unsound; I've opened #19 about that.

More generally, to make this sound you need to write either to a Vec without using set_len() or to &mut MaybeUninit to make things sound and fast. Writing to a slice would require initializing it first, which in case of something as fast as LZ4 would cut performance in half.

Shnatsel commented 3 years ago

Well, I guess writing to the Vec's capacity through raw pointers and only then calling set_len() would also work, provided you're careful about aliasing.