bytecodealliance / sightglass

A benchmark suite and tool to compare different implementations of the same primitives.
Apache License 2.0
69 stars 33 forks source link

Removes define of NDEBUG to allow asserts to work #244

Closed jlb6740 closed 1 year ago

abrown commented 1 year ago

The benchmarks/run CI task should be green if we rebase this on #243. But to get the benchmarks/rebuild task to work I think we may need to rebuild all of the benchmarks locally and check them back in to Git. Changing how we build the benchmarks will change the output WebAssembly and that is what that benchmarks/rebuild is doing--ensuring we build the same bytes in CI as we would locally.

abrown commented 1 year ago

(or maybe ripping out NDEBUG doesn't affect the WebAssembly output?!?)

jlb6740 commented 1 year ago

Yes, if we modify the dockerfile used to build the benchmarks then all the wasm benchmarks that use that file should be checked in as well. I can submit a patch on top of this one that does that.

jlb6740 commented 1 year ago

Note, after removing NDEBUG the assert message is as follows.

Running `target/debug/sightglass-cli benchmark -d /home/jlbirch/sightglass-jlb6740/benchmarks/shootout-ackermann/ --processes 1 --iterations-per-process 1 --engine engines/wasmtime/libengine.so -- benchmarks/shootout-ackermann/benchmark.wasm`

failed to execute

Caused by: 0: error while executing at wasm backtrace: 0: 0x2a72 - !abort 1: 0x2e8f - !assert_fail 2: 0x39b - !read_int_from_file 3: 0x3f7 - !original_main 4: 0x292 - !_start note: using the WASMTIME_BACKTRACE_DETAILS=1 environment variable may show more debugging information 1: wasm trap: wasm unreachable instruction executed thread 'main' panicked at 'assertion failed: (left == right) left: -1, right: 0', /home/jlbirch/sightglass-jlb6740/crates/recorder/src/bench_api.rs:250:9 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace"

Note the above does not write to the stderr, but this does:

jlbirch@jlbirch-machine 20230608 09:50:18 ~/sightglass-jlb6740: more stderr-2cfe3c2c79b5e6e-1417880-0.log Assertion failed: fd != -1 (benchmark.c: read_int_from_file: 25)

jlb6740 commented 1 year ago

Issue first discovered #238

abrown commented 1 year ago

Rebase?