search

rust-fuzz / book

📖 Guides and tutorials on how to fuzz Rust code
https://rust-fuzz.github.io/book/
132 stars 22 forks source link

Coverage report information is very out of date #39

Open mumbleskates opened 9 months ago

mumbleskates commented 9 months ago

The commands described in the coverage page for generating a coverage report, or doing anything at all with the profdata files collected, are obsolete; none of the replacement or alternative commands seem to accept those arguments. For example, cargo-cov seemingly no longer has a "show" command at all.

one command that does seem to work is

llvm-cov show --format=html -Xdemangler=rustfilt \
 --instr-profile=fuzz/coverage/fuzz_target_binary/coverage.profdata \
 target/x86_64-unknown-linux-gnu/coverage/x86_64-unknown-linux-gnu/release/fuzz_target_binary \
 . \
 > coverage.html

but for various reasons this might not be ideal.

cameronelliott commented 9 months ago

I'm not a maintainer on this project, but I just went through the tutorials twice myself. Once on macos, and once on linux.

I ran into a similar issue.

I think, but am not certain that the issue you ran into, is that the cargo cov problem you ran into is not having the correct cargo subcommand installed.

The correct way I set this up was like this cargo install cargo-binutils.

I did actually install the wrong subcommand cargo install cargo-cov, which caused me to dig into my working rust-fuzz tutorial macos setup to discover I needed cargo-binutils not cargo-cov.

With that subcommand install, the tutorial works fine, with one other tweak I will file an issue on. (you need to tweak the path to the binary) The rust-fuzz book coverage page should explicitly mention needing cargo install cargo-binutils in order for the cargo cov to launch the correct subcommand. The rust-fuzz book coverage page does mention "installing LLVM-coverage tools", and on the linked rustc book page it causually mentions "In this case, you may also find cargo-binutils useful as a wrapper around these tools." But IMHO it's not enough...

I will file an issue.

mumbleskates commented 9 months ago

yeah, i already have LLVM-coverage tools installed. i have the whole LLVM-19 suite, so i was eventually able to figure out getting a coverage report directly with the underlying tools, as in the above example...

that's good to know that there is a different cargo subcommand entirely, in cargo-binutils. cargo search doesn't even find that one and it doesn't seem to be mentioned anywhere.