search

mcmilk / 7-Zip-zstd

7-Zip with support for Brotli, Fast-LZMA2, Lizard, LZ4, LZ5 and Zstandard
https://mcmilk.de/projects/7-Zip-zstd/
Other
5.01k stars 299 forks source link

Benchmark in Doku is quite old. #208

Open stefano2734 opened 3 years ago

stefano2734 commented 3 years ago

And with the new features and changes in zstd 1.5, there is some interest by results and possible regressions.

The documented benchmark is excellent, but perhaps quite long for all tests. An automatic benchmark script with comparison to older versions is also a part of a good quality system.

Here is an example of a newer test. https://bestofcpp.com/repo/facebook-zstd-cpp-compression

mcmilk commented 3 years ago

Yes, but currently I am short of time ... I wanted to update the benchmark in 2020 already :/

HarvestSnowpaw commented 2 years ago

Here's a start. I made an updated ratio vs speed chart, running benchmarks against the "generate-mcorpus" script that was used on the original, with version 1.5.0R2 x64 for a selection of the included compressors. I'd be glad to help with the rest of the benchmarks, just need a little time to set up the scripts & graphs.

Compression Ratio vs Speed

stefano2734 commented 2 years ago

Thanks for this result. The scale is here not linear at x and tricky. so I see here compression like bzip2 with speed of lz4 for zstd. Only brotli is here near. Old Standard Deflate in most business and government is slow and with more space.

In 1.5.1 there are some performance improvements with 1.5.2 only bugfix. Also 1.5.5 have some improvements. So a new run with 1.5.5 is on the road for better results of zstd by new release.

dreamflasher commented 1 year ago

@mcmilk Thank you for your great work on this project. I'd also be very happy if you'd update the benchmarks :)

mcmilk commented 1 year ago

I do not have a windows pc in the moment.

stefano2734 commented 1 year ago

More than 2 years after begin of this issue the benchmark should run. Open Test by users can solve the problems of the author with time and equipment. How about a hardware contest with different new and older hardware by amd, Intel and others? Harvestsnowpaw made a test with 1.5.0 two years ago. So it is not impossible. issue #320 shows here weakness in code of 7zip. Benchmark is here a necessary detection tool.

HarvestSnowpaw commented 11 months ago

When I get a chance, I'll update the benchmarks I did with the newest version of 7z-zstd. Thank you @stefano2734 for the note about the scale not being linear. The problem is that there is such a range of data, that otherwise the chart would be massive to account for all the speeds. I will still take a look and see if there's any way I can clean it up.

stefano2734 commented 11 months ago

When I get a chance, I'll update the benchmarks I did with the newest version of 7z-zstd. Thank you @stefano2734 for the note about the scale not being linear. The problem is that there is such a range of data, that otherwise the chart would be massive to account for all the speeds. I will still take a look and see if there's any way I can clean it up.

With non-linear scale, the user need an ideal line or milestones. By multiprocessor there is an ideal curve of perfect scaling. And by nonlinear scale also value lines of double or factor 4 can make better understanding. /

stefano2734 commented 5 months ago

In Zstandard there is a actual benchmark with zstandard 1.5.6 and other compression codecs.

dreamflasher commented 5 months ago

The mentioned benchmarks can be found here: https://github.com/facebook/zstd?tab=readme-ov-file#benchmarks – nice! Unfortunately, they are missing fastlzma2, lzlib, lzma).

They in return reference lzbench: https://github.com/inikep/lzbench – which has a bunch of additional libs, but is unfortunately outdated for zstd.

stefano2734 commented 1 month ago

In GitHub of zstandard actual values to version 1.5.6 made with lzbench

https://github.com/facebook/zstd