search

kekyo / PowerPlayZipper

An implementation of Lightning-Fast Zip file compression/decompression library on .NET.
Apache License 2.0
13 stars 1 forks source link

No improvement over `System.IO.Compression.ZipArchive` #1

Open A9G-Data-Droid opened 1 year ago

A9G-Data-Droid commented 1 year ago

I am rapidly dealing with zip files in the range of 250MB. I am seeing no improvement over my old System.IO.Compression.ZipArchive method. After running a few times I am reliably seeing the same results, a bit slower.

System.IO.Compression.ZipArchive    Runtime: 00:00:01.6661219
PowerPlayZipper                     Runtime: 00:00:01.6874230
kekyo commented 1 year ago

Thanks for trying it out!

PowerPlayZipper speeds up the process by decompressing large numbers of files in parallel. Are you saying that the zip file you tried contains a single 250MB file?

A9G-Data-Droid commented 1 year ago

That makes sense. Yeah, maybe this archive is not a good candidate for parallelization.

kekyo commented 1 year ago

In fact, it would be nice to be able to achieve parallel decompression of a single file. I am not familiar with compression algorithms themselves, so I am not clear, but to achieve parallel processing with a compression algorithm, I think we need to design a suitable algorithm from the beginning.

Perhaps "Deflate" is an old algorithm, so even if we add some thought to our implementation, we can't expect it to be fast. And PowerPlayZipper depends on System.IO.Compression. So it will be rate-limiting in a single file.