maxi-k / btrblocks

BtrBlocks: Efficient Columnar Compression for Data Lakes (SIGMOD 2023 Paper)
MIT License
217 stars 18 forks source link

option to set blocksize #4

Open seb711 opened 6 months ago

seb711 commented 6 months ago

Added an option to set the blocksize used in btrblocks based on the analysis made. With that the default Blocksize is still 1 << 16. But as the analysis showed a better performance could be achieved by reducing the blocksize. To better evaluate this analysis I've modified the decompression-speed script; that on the one hand now measures the correct decompressed size and on the other hand can verify the result with the binary uncompressed data.