META: benchmarks to include

[anp]

This is a tracking issue for new benchmarks we'd like to add. If you're landing here from hacktoberfest (welcome!), feel free to see the project README for more information :D.

The basic process goes:

1. find where the project in question keeps its benchmarks (usually in its repo under benches)
2. read the documentation on adding new benchmarks
3. add the benchmarks
4. send a PR
5. at a point in the near future, we'll start seeing results from the benchmarks you ported on https://blog.anp/lol/lolbench-data

unclaimed

please comment on individual issues to claim them!

• aho-corasick
• futures
• backtrace
• smallvec/vecmap
• enjarify
• an arena allocator
• backtrace
• claxon
• benchmark-game
• port ISPC examples
• port SPEC
• latest inflate

finished

• ~csv~ thanks @shssoichiro!
• ~byteorder~ thanks @thomwiggers!
• ~RustCrypto~ thanks @thomwiggers!
• ~clap~ thanks @kitlith!

[anp]

https://github.com/Storyyeller/enjarify/tree/go/enjarify-rs ?

[anp]

https://benchmarksgame.alioth.debian.org/u64q/rust.html ?

[anp]

https://github.com/ruuda/claxon ?

[gnzlbg]

We should probably also include the benchmark-game programs from https://github.com/TeXitoi/benchmarksgame-rs .

Also I always wanted to port the remaining examples of the rust-lang-nursery/simd crate to rust-lang-nursery/stdsimd, maybe we should include those once they are done. They will probably both have a scalar and vectorized implementation, but catching regressions on LLVM vector type optimizations is worth pursuing.

The ISPC compiler has a bunch of examples that we might want to port to Rust: https://github.com/ispc/ispc/tree/master/examples

From the standard suites, porting SPEC2017 (https://www.spec.org/benchmarks.html) to Rust.

---

IMO when porting thins, we should port benchmarks to "idiomatic" Rust code.

[anp]

I agree! :p

Right now the benchmark count bottleneck is having enough machines to run them in a timely fashion, I need to dig in more on the bench timings and get the new server(s) I'm setting up ready.

I think ISPC and SPEC benchmarks would be fantastic to track as well!

[anp]

per @eddyb on irc, we should also include an equivalent to the inflate 0.3.4 benchmark with a more recent version of the crate.

[eddyb]

The very important bit is whether or not the debloating https://github.com/PistonDevelopers/inflate/pull/35 is included.

[anp]

@eternaleye gave me some reading homework and I'm saving it here for future reference:

https://www2.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-183.html

https://algowiki-project.org/en/Open_Encyclopedia_of_Parallel_Algorithmic_Features

http://superfri.org/superfri/article/view/69

https://github.com/vtsynergy/OpenDwarfs

http://dsc.soic.indiana.edu/publications/OgreFacets.pdf

https://www2.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-144.html

https://www.sciencedirect.com/science/article/pii/S1877050912003353

[gnzlbg]

I've uploaded the ambient occlusion (ray-tracing) benchmarks: https://github.com/gnzlbg/aobench

The readme shows how to run the scalar, vectorized, and parallel versions. Probably it only makes sense to add the scalar and scalar_par benchmarks initially. The vector and vector_par benchmarks use std::simd and that's still in flux.

[thomwiggers]

https://benchmarksgame.alioth.debian.org/u64q/rust.html ?

Alioth is dead, but see https://benchmarksgame-team.pages.debian.net/benchmarksgame/measurements/rust.html

[newpavlov]

How about some crates from RustCrypto? sha2, sha3 and one of AES crates can be a good start.

[anp]

Going to lock this so it can just be a tracking issue. If you have ideas for other benchmarks we should include, please open an issue!