search

guzba / zippy

Pure Nim implementation of deflate, zlib, gzip and zip.
MIT License
258 stars 31 forks source link

incremental gzip uncompress #91

Open chancyk opened 1 month ago

chancyk commented 1 month ago

I needed to incrementally decompress very large (~100GB) gzipped JSON files. This is working for me and seems to only be about 20% slower than the one-shot version for large files, but 6x slower for small files, so probably worth leaving it as a separate API.

guzba commented 1 month ago

Thanks for the PR. I am traveling until ~the end of the month so I'll be delayed in looking closer at this. I think the idea of having a way to decompress large files is very reasonable. I want to look at and think through the details here when I am able.