bytecodealliance / sightglass

A benchmark suite and tool to compare different implementations of the same primitives.
Apache License 2.0
69 stars 33 forks source link

Add `libsodium` benchmarks #252

Closed abrown closed 1 year ago

abrown commented 1 year ago

This change builds all of the libsodium benchmarks into WebAssembly files compatible with Sightglass' bench_start() and bench_end() markers. Two of the benchmarks, libsodium-sodium_utils2.wasm and libsodium-sodium_utils3.wasm, are compiled with the bench_end() call for some reason and are skipped for now.

abrown commented 1 year ago

This depends on #251; do not merge this until that is merged. I'm putting this up as a draft to give a concrete example of what #251 for a multi-file benchmark and to make sure CI passes.

abrown commented 1 year ago

This is ready for review and merging!

abrown commented 1 year ago

It looks like the benchmarks are printing things -- can we get expected stdout and/or stderr files to make sure that the benchmarks continue to run correctly in the future?

The std{out|err}.expected pattern may need to change if we dump more than one .wasm file in a directory. Maybe we can figure that out in a separate PR. Some logic like, "if we find stdout.expected, use that; if not, look for <benchmark-name>.stdout.expected". What do you think of that?

fitzgen commented 1 year ago

I'm fine with changing everything to <benchmark-name-without-extension>.std{out,err}.expected, so all the existing ones would be benchmark.std{out,err}.expected and then libsodium can do its multi-benchmark thing too.

jedisct1 commented 1 year ago

sodium_utils2 and sodium_utils3 require memory protection, so on wasm32-wasi, they abort immediately. It's not worth including them in benchmarks.

abrown commented 1 year ago

Ok, I'm going to merge this as-is. I opened #256 to handle the <benchmark-name-without-extension>.std{out,err}.expected change so I'll handle that there. I think we may want to drop the number of compiled-in libsodium iterations down from 200 to 10 or 1 but I may handle that in #254.