search

bitcoinknots / bitcoin

Bitcoin Knots enhanced Bitcoin node/wallet software
MIT License
219 stars 65 forks source link

Feature Request: Support for GZip Compressed Block Files #6

Open rmackinnon opened 7 years ago

rmackinnon commented 7 years ago

User story: As a user I will minimize the used disk space for block files by utilizing a compression algorithm (Gzip, BZ2, LZMA, 7zip, etc) to compress and store the blocks when not in use. When needed for comparison, they are decompressed and checked. Whereas the most recent block would remain uncompressed for appending.

I ran some compression tests on the current block chain set can and found a storage savings of approximately 22GB, with compression ratios from 15.9% (blk00749.dat) to 43.0%(blk00295.dat). This is a significant enough savings that may warrant compression on older blocks. I did not find any significant difference between the various compression algorithms, most were within 1-2MiB of each other.

Lowest Compression: blk00749.dat 15.9% Uncompressed Size: 134089259 Compressed Size: 112736277 Algorithm Used: Gzip

Highest Compression: blk00295.dat 43.0% Uncompressed Size: 133492866 Compressed Size: 76137593 Algorithm Used: Gzip

Average Compression Ratio Across Entire Set: 20.7%

luke-jr commented 7 years ago

For now, I suggest using a filesystem that supports compression, such as btrfs.