search

lemire / simdcomp

A simple C library for compressing lists of integers using binary packing
BSD 3-Clause "New" or "Revised" License
488 stars 53 forks source link

Use this to compress file? #25

Closed raspiduino closed 1 year ago

raspiduino commented 1 year ago

If I read a file, then divide the file into blocks, each blocks is an array of uint32_t, then I use the library to compress that uint32_t array, then will it be efficient?

Also, what happens if I repeatedly compress the output, how far would I go before the output size >= the input size? I know there should be a limit

Also, in the readme:

The assumption is either that you have a list of 32-bit integers where most of them are small

What is the limit of small?

Thank you!

lemire commented 1 year ago

Thanks for the questions. If you feel like the library is the right tool for you, I recommend you put it to the test in your context. Then if you encounter bugs or lack of documentation, please open an issue.

The library is a low-level library that requires you to design your own binary formats.