haf / DotNetZip.Semverd

Please use System.IO.Compression! A fork of the DotNetZip project without signing with a solution that compiles cleanly. This project aims to follow semver to avoid versioning conflicts. DotNetZip is a FAST, FREE class library and toolset for manipulating zip files. Use VB, C# or any .NET language to easily create, extract, or update zip files.
Other
545 stars 218 forks source link

Performance improvement for zipping with AES encryption #273

Closed AndrewBSzabo closed 1 year ago

AndrewBSzabo commented 1 year ago

I left an issue about this (#271), but these changes will fix the issue.

These were the results I got from some performance testing: image

I tested with varying the encryption types (PKzipWeak vs WinZipAes256) with different compression types (Level9, Level4, and Level0) when compressing an 86.9MB folder with 2282 files. When using WinZipAes256 encryption (bottom 3 rows) we can see that after the performance changes made by this commit (last column) we saw ~85-73% (depending on the compression level) reduction in runtime compared to before the performance changes made by this commit (second last column).

If you need me to clean up the code at all before merging these changes, please let me know. Thanks!

AndrewBSzabo commented 1 year ago
  1. I modified the code so that the passwords of the ZipEntry objects are used. I then will store encryption keys that have already been calculated in a dictionary (Dictionary<Password,EncryptionKey>) so that if the same password is used, we can just grab it from the dictionary rather than recalculating it again. With this change the performance improvement is now dependent on how many different passwords are being used when creating/adding to a zip. The following screenshot are some performance numbers when the number of passwords used in a file is varied from 1, 10, 100, and 1000: image We can see that we get the largest performance improvement when using <10 passwords, and the smallest performance improvement when using >1000 passwords (still some improvement, just not as much).

  2. Fixed the build issue

  3. All related unit tests are still passing, and I added a new WinZipAes test for creating a zip with entries that have different passwords.

AndrewBSzabo commented 1 year ago

Corrected this error and added two more unit tests. Let me know if there are any other issues. Thanks!

AndrewBSzabo commented 1 year ago

made those changes.