bytecodealliance / sightglass

A benchmark suite and tool to compare different implementations of the same primitives.
Apache License 2.0
69 stars 33 forks source link

Fix shootout-ackermann #243

Closed abrown closed 1 year ago

abrown commented 1 year ago

As reported in #238, the shootout-ackermann benchmark output did not match what was expected. While debugging this, several things became apparent:

Both of these issues seem wasi-libc related and might warrant some investigation there. This benchmark currently is building itself with wasi-sdk v19.0; by reverting this commit, here are the steps to replicate this issue:

$ benchmarks/build.sh benchmarks/shootout-ackermann/
$ RUST_LOG=trace cargo run -- benchmark --engine=engines/wasmtime/libengine.so --processes 1 --iterations-per-process 1 benchmarks/shootout-ackermann/benchmark.wasm

In any case, using O_RDONLY seems to fix the issue so we can close

238 and continue the discussion in the wasi-libc repository.

jlb6740 commented 1 year ago

Great find Andrew!

abrown commented 1 year ago

After some discussion today, it became clear that at least the assert issue is of our own doing: at some point we "undefined" NDEBUG which changes how assert works in C generally. https://github.com/bytecodealliance/sightglass/pull/244 fixes this.