search

tsolomko / SWCompression

A Swift framework for working with compression, archives and containers.
MIT License
233 stars 39 forks source link

A sample of bz2 file that takes too long to decode (7mb -> 1min) #10

Closed Lakr233 closed 4 years ago

Lakr233 commented 4 years ago

Sample File: Packages.bz2.zip

Programme Behave: 截屏2020-04-26下午9 48 49

I have no idea why this is happening............ :/

tsolomko commented 4 years ago

I have slightly unrelated question: what have you used to create your ZIP file?

Lakr233 commented 4 years ago

I have slightly unrelated question: what have you used to create your ZIP file?

I used the Finder to create. (that right-click

btw just be clear, GitHub does not allow me to upload .bz2 file directly so I uploaded a zip of that bz2

tsolomko commented 4 years ago

Your ZIP file actually helped me uncover an interesting issue. Apparently, Finder, or whatever it is using under the hood, produces ZIP files which don't strictly conform to ZIP specifications (Extended Timestamp Extra Field in the Central Directory must only contain modification time, but Finder also puts there access time and creation time). What's even more interesting is that Finder in my version of macOS produces correct ZIP files.

Currently, SWCompression crashes when trying to open such incorrect ZIP files. While I think that this is, generally, an issue on Finder's part and should be fixed by Apple, at the same time SWCompression probably shouldn't crash when it encounters these ZIP files. I am going to fix this particular issue in the near future.

Anyway, back to your original question. I was unable to reproduce the problem you've encountered: on my pretty old machine (Macbook Pro Late 2011) with macOS 10.13 and Swift 5.0.3 it took SWCompression (built in Release mode) less than 2 seconds to decompress your BZip2 file. Which leads me to the following questions:

  1. Are you building SWCompression in Debug or Release mode? Please note, that the optimizations Swift compiler does in Release mode dramatically improve the performance of SWCompression.

  2. What Swift version are you using?

Lakr233 commented 4 years ago

Are you building SWCompression in Debug or Release mode?

Currently debut mode. I will pay attention to this tricky switch next time.

What Swift version are you using?

Swift 5.1? I used latest Xcode with it’s SDKs.

——— Currently, debug sessions are reporting decode times from 30s to 350s floating between them while other files with same type can do a pretty job. I still having no idea why this is happening. I’ll follow up this post when I got some idea.

———

Your ZIP file actually helped me uncover an interesting issue

The zip file format was badly played by coders around the world, fact is that command line such as unzip (usually from apt or home brew) cannot handle all of them. There are always zip files with extra info attached to or with illegal byte sequence. ! Been a headache over years 🤦‍♂️

Lakr233 commented 4 years ago

This bug is gone silently in release mode.