Open M-Gonzalo opened 4 years ago
Weighted average speedup for fxz: 195%
Weighted average speedup for fxz -e: 168%
Weighted average ratio gain for fxz: 2.03%
Weighted average ratio gain for fxz -e: 2.20%
Speed = (size of file.pcf_cn) / (time of [ precomp || fxz ])
Ratio = (size of file) / (size of file.pcf_cn.[ precomp || fxz ])
Sets:
* zpaqgui-win32.win32.x86_64.zip
* APKs.zip, which is just a few Android apps in a zip
-both after "precomp -cn -intense -brute -t-3 -packjpg-" (because of ARM packjpg and packmp3 bugs)
Machines:
Android linux, Termux environment, 8-core ARM Cortex A53, 4gb RAM
Manjaro linux, 4 logical threads Intel Atom, 2gb RAM
Ubuntu linux, 2-core Intel Xeon, 16gb RAM
Methods:
precomp = precomp -t+ -e file.in
fxz= fxz -z -9 -T0 -c file.in >file.in.fxz
fxz 'extreme'= fxz -z -9 -T0 -c -e file.in >file.in.fxze
=========================================================== zpaqgui-win32.win32.x86_64.zip (pre-compresed):
On ARM Cortex A53, 8 threads:
55.83% - 2906 kbps => fxz 'extreme' [185% faster, 1.4% stronger]
56.00% - 3148 kbps => fxz [200% faster, 1.2% stronger]
57.22% - 1574 kbps => precomp
On x86-64 Intel Atom, 4 threads:
55.89% - 1095 kbps => fxz 'extreme' [184% faster, 0.65% stronger]
56.03% - 1281 kbps => fxz [215% faster, 0.51% stronger]
56.54% - 595 kbps => precomp
On x86-64 Intel Xeon, 2 threads:
55.87% - 3675 kbps => fxz 'extreme' [223% faster, 0.01% stronger]
55.88% - 1650 kbps => precomp
56.03% - 4264 kbps => fxz [258% faster, 0.45% weaker]
=========================================================== APKs.zip (pre-compresed)
On ARM Cortex A53, 8 threads:
63.09% - 3193 kbps => fxz 'extreme' [119% faster, 9.8% stronger]
64.06% - 3721 kbps => fxz [139% faster, 8.8% stronger]
72.89% - 2686 kbps => precomp
On x86-64 Intel Atom, 4 threads:
63.89% - 1244 kbps => fxz 'extreme' [161% faster, 7.32% stronger]
64.05% - 1502 kbps => fxz [195% faster, 7.16% stronger]
71.21% - 772 kbps => precomp
On x86-64 Intel Xeon, 2 threads:
63.88% - 4181 kbps => fxz 'extreme' [173% faster, 2.85% stronger]
64.03% - 4946 kbps => fxz [205% faster, 2.70% stronger]
66.73% - 2413 kbps => precomp
===========================================================
These are only two datasets that are representative of several other more informal tests during the past months. BTW: fxz uses ~900-1000m RAM on the default configuration, instead of the ~1300 that precomp does, although it can be configured to use a bigger dictionary and, in doing so, provide even better compression.
I've done some tests to determine whether Conor's fastlzma2 library could actually benefit precomp. The answer is yes and by a long shot. The tests have proven that the novel algorithm provides about 2x faster compression, all while compressing up to 10% more and using less memory than multi-threaded lzma. So it's basically double speed for free.
Because of this...
... the difference on ratio is more visible when working on a high number of threads, which on mt-lzma usually means chunking the input and losing long range compression, but with flzma2 doesn't.