search

FuelLabs / fuel-vm

Fuel v2 interpreter in Rust
Other
358 stars 88 forks source link

Investigate missing fuzz coverage reports in CI #856

Closed netrome closed 1 month ago

netrome commented 1 month ago

Context

While the CI fuzzing seems to work in the sense that it runs the fuzzers on PRs and has successfully found injected errors, we don't seem to get the expected coverage reports as can be observed in https://github.com/FuelLabs/fuel-fuzzing-corpus/commit/bf288985f3b94e8048d36b457c750cf7af6865b5

We should investigate and understand better why these reports aren't uploaded.

Definition of done

The reason for the missing coverage reports is understood.

netrome commented 1 month ago

Great success!

https://github.com/FuelLabs/fuel-vm/actions/runs/11381732377/job/31663774675

The problem seems to have been this error which I got locally but wasn't shown in the CI runs for some reason:

INFO:__main__:Running: docker run --privileged --shm-size=2g --platform linux/amd64 --rm -i -e FUZZING_ENGINE=libfuzzer -e HELPER=True -e FUZZING_LANGUAGE=rust -e PROJECT=fuel-vm -e SANITIZER=coverage -e 'COVERAGE_EXTRA_ARGS= ' -e ARCHITECTURE=x86_64 -e HTTP_PORT=8008 -p 8008:8008 -v /tmp/corpus/grammar_aware_advanced_corpus:/corpus/grammar_aware_advanced -v /home/marten/fuel/x/oss-fuzz/build/out/fuel-vm:/out -t [gcr.io/oss-fuzz-base/base-runner](http://gcr.io/oss-fuzz-base/base-runner) coverage grammar_aware_advanced.
Running grammar_aware_advanced
warning: /out/dumps/grammar_aware_advanced.15203446424991361819_0.profraw: raw profile version mismatch: Profile uses raw profile format version = 10; expected version = 9
PLEASE update this tool to version in the raw profile, or regenerate raw profile with expected version.
error: no profile can be merged
[2024-10-17 08:01:59,940 INFO] Finding shared libraries for targets (if any).
[2024-10-17 08:01:59,946 INFO] Finished finding shared libraries for targets.
error: /out/dumps/grammar_aware_advanced.profdata: could not read profile data!No such file or directory
error: /out/dumps/grammar_aware_advanced.profdata: could not read profile data!No such file or directory
error: /out/dumps/*.profdata: No such file or directory
ERROR:__main__:Failed to generate clang code coverage report.

So the coverage generation didn't work due to a version mismatch in the profile file. Removing the pinned Rust compiler version eliminated this issue, and is better aligned with the docs in https://google.github.io/oss-fuzz/getting-started/new-project-guide/rust-lang/#dockerfile.