MacPaw / XADMaster

Objective-C library for archive and file unarchiving and extraction
GNU Lesser General Public License v2.1
105 stars 45 forks source link

[LSAR/UNAR] Slow text output #121

Open kyeastmood opened 4 years ago

kyeastmood commented 4 years ago

Archive: link This archive contains 100K files and it can show the difference in performance between LSAR/UNAR and 7-Zip when printing text. Archives containing tens of thousands of files are not unusual.

Results of listing contents of the archive verbosely:

7-Zip:

$ time 7z l -slt 100k_files.7z >> 7zip.log

real    0m0.981s
user    0m0.572s
sys     0m0.403s

Time: ~1 second Log size: 16 MiB

LSAR:

$ time lsar -L 100k_files.7z >> lsar.log

real    1m5.481s
user    1m5.009s
sys     0m0.412s

Time: 1 minute 5 seconds Log size: 41.7 MiB

So we can see that LSAR listed the archive about 60 times slower but the output log is only less then 3 times bigger. This is a significant bottleneck even when processing smaller archives containing only 10-20K files.

Thanks

P.S. Sorry for spamming you with quite a few issues at the same time. I've accumulated them over the last ~4 years of usage and I've learned only recently that you have a repository on GitHub.

MaddTheSane commented 4 years ago

Perhaps in areas which use mutable types, the creation could pre-allocate the capacity. An example: https://github.com/MaddTheSane/XADMaster-2/commit/056bd4209d6e6eb8a6c11b622bbc1e2345d5ae74

MaddTheSane commented 4 years ago

Running this through my own fork through Instruments, it looks like a lot of it is taken up by regncomp_l (30.6%). Maybe caching the regex could cut down on time (IIRC, Colloquy does that)?