search

stacks-network / clarity-wasm

`clar2wasm` is a compiler for generating WebAssembly from Clarity.
GNU General Public License v3.0
14 stars 13 forks source link

Property test is passing 'cargo test' but failing in coverage #331

Closed ameeratgithub closed 6 months ago

ameeratgithub commented 8 months ago

Property test for sequences::len_crosscheck passes CI and cargo test, but it fails in the coverage check. For details, please visit the coverage report.

csgui commented 8 months ago

Hey @ameeratgithub !

Today the steps for test and coverage are disconnected, meaning that both run the set of tests separately. We have changes on the way to make those 2 jobs more related.

Meanwhile, can you please check on your local, the output of the following command?

cargo llvm-cov --all-features --workspace --codecov --output-path codecov.json

Please let me know the output. Tks!

ameeratgithub commented 8 months ago

Sometimes it runs successfully, sometimes it throws error because of random value in property test. I ran the property test, sequences::len_crosscheck, in a loop and found the snippet which is causing the problem. I'll assign the issue to myself. Error is related to memory. I'm just leaving the error message here

assertion `left == right` failed: Compiled and interpreted results diverge!
compiled: Err(Wasm(Runtime(error while executing at wasm backtrace:
    0: 0x126ed - <unknown>!<wasm function 53>

Caused by:
    0: memory fault at wasm address 0x10000 in linear memory of size 0x10000
    1: wasm trap: out of bounds memory access)))
interpreted: Ok(Some(UInt(12)))
  left: Err(())
 right: Ok(Some(UInt(12)))
csgui commented 8 months ago

Yep.. the issue is with the randomness of value ranges. That step can be successful or not.

smcclellan commented 7 months ago

This will be fixed when https://github.com/stacks-network/clarity-wasm/issues/113 is addressed.