search

bytecodealliance / sightglass

A benchmark suite and tool to compare different implementations of the same primitives.
Apache License 2.0
69 stars 33 forks source link

Build Rust benchmarks from a common `Dockerfile` #248

Closed abrown closed 1 year ago

abrown commented 1 year ago

Emscripten- and wasi-sdk-built projects already use a common Dockerfile for rebuilding the benchmarks. This change extends that to the Rust-built benchmarks by symlink-ing to Dockerfile.rust. The idea is to reduce duplication and make it easier to change build configuration in one place. For example, this change also bumps the used version of Rust to 1.70, which causes the emitted WebAssembly to slightly change.

jlb6740 commented 1 year ago

I can not reproduce the hex-simd failure with CI, can you? Seeing that we don't have access to those log files, in situations like this it may be nice to print the diff between the expected output and the actual output.

abrown commented 1 year ago

After talking to @alexcrichton about hex-simd, I pushed 3c061ec to make the benchmark more robust to changes in the WebAssembly binary. I think we can safely ignore the rebuild failure: it's spidermonkey, which can't rebuild itself (#231).

fitzgen commented 1 year ago

I think we can safely ignore the rebuild failure: it's spidermonkey, which can't rebuild itself (#231).

Can we make CI green by ignoring spidermonkey rebuilds or something here? I don't want to get used to CI always being red because "CI isn't passing because X reason" and then that hides Y regression that we don't notice from landing.