Open afonso360 opened 2 years ago
https://github.com/bytecodealliance/wasmtime/blob/9cb987c678f93e7d30fb702aece7dad641a0f007/cranelift/src/bugpoint.rs#L986-L1041 is where we determine if it is still considered a crash. If first checks for verifier errors and if so considers the crash to have gone away (so we don't try to compile functions that are invalid) and then check that compilation didn't panic at https://github.com/bytecodealliance/wasmtime/blob/9cb987c678f93e7d30fb702aece7dad641a0f007/cranelift/src/bugpoint.rs#L1029-L1036. This last block could be changed to test if repros still crash at runtime.
👋 Hey,
Feature
bugpoint is a really interesting tool if some clif code is failing to compile. Some of outputs reported by the clif differential fuzzer don't fail to compile, but predictably crash during execution. We could expand bugpoint to iterate while the run keeps crashing.
Benefit
This would help a lot minimizing the test cases dumped by the clif fuzzer. The fuzzing engine isn't very good at minimizing those cases.
Implementation
I'm not too familiar with bugpoint, but I guess we can add a mode that besides compiling does something similar to what
clif-util test
does. We would have to catch panics and exceptions thrown during execution and probably ensure that they don't change while iterating.Alternatives
Minimizing by hand is kinda painful but we can keep doing it.