bytecodealliance / wasmtime

A fast and secure runtime for WebAssembly
https://wasmtime.dev/
Apache License 2.0
15.45k stars 1.3k forks source link

Improve fuzzing of `*.wast` tests #9587

Closed alexcrichton closed 2 weeks ago

alexcrichton commented 2 weeks ago

Currently we have a fuzzer which is tasked with running *.wast tests with fuzz-generated configurations. This asserts that we at least satisfy all basic wasm semantics regardless of how various knobs in Config are turned (modulo limits to resources). The current fuzzing though is not comprehensive in that it doesn't include all the spec tests that we pass from all proposals. This runs the risk of we don't actually fuzz anything until the spec tests are merged upstream, which can take a significant amount of time.

This commit refactors the *.wast-management infrastructure to share test discovery and feature calculation between tests/wast.rs and fuzzing. This new support crate centralizes limits and discovery for both to use. Additionally fuzzing is updated to no longer throw out test cases if configuration isn't applicable but instead clamp configuration to the minimum required values (e.g. features + resource limits). This means that we should now be fuzzing all spec tests that pass in all configurations.

This new fuzzer discovered a few minor issues with the GC proposal implementation, for example, such as:

github-actions[bot] commented 2 weeks ago

Subscribe to Label Action

cc @fitzgen

This issue or pull request has been labeled: "fuzzing", "wasmtime:ref-types" Thus the following users have been cc'd because of the following labels: * fitzgen: fuzzing, wasmtime:ref-types To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file. [Learn more.](https://github.com/bytecodealliance/subscribe-to-label-action)