PSeitz / lz4_flex

Fastest pure Rust implementation of LZ4 compression/decompression.
MIT License
460 stars 31 forks source link

Public API for compressing with prepended size but providing my own buffer for memory re-use #166

Open EriKWDev opened 3 months ago

EriKWDev commented 3 months ago

would be nice if there was a

let size = lz4_flex::block::get_maximum_output_size_prependend_size();
let mut output = vec![0_u8; size];
lz4_flex::compress_prepend_size_into(&data, &mut output);

Internally, lz4_flex already does

let prepend_size_num_bytes = if prepend_size { 4 } else { 0 };
let max_compressed_size = get_maximum_output_size(input.len()) + prepend_size_num_bytes;
// ...
let mut compressed: Vec<u8> = vec![0u8; max_compressed_size]; // <- This one with 'safe'decode'
// ...
let mut vec = Vec::with_capacity(max_compressed_size); // <- This one otherwise

Would be nice if this allocation could be up to the user instead so that I can re-use it for multiple compressions of packets in my TCP server without re-allocating it every time.

Ofc, with a matching decompress_size_prepended_into.

It would be fine by me to have some demands on the user of such an API such as require filling it with 0:s or not for the safe-decode ct-branch. The function could then be wrapped in a helper that does that, but I just want this functionality exposed publicly.

It's sad when useful code gets put behind private rust API:s and I keep having to make tiny forks.... happens waaay too often with libraries in this language which is why I always make everything public and only document the intended stuff. I much prefer putting the 'scary private' functions in a small module hidden in the docs but at least available to users willing to have to learn the internals.

pub(crate) is the bane of my existence xD