provide for cargo afl, but it is not complete.

[xizheyin]

Hi! I've added initial support for cargo afl, but it's not perfect. I'm not quite sure what the norms are for contributing code, and to what extent it can be merged.

Running cargo llvm-cov afl will run afl for 10 seconds (I'll modify it later to let the user set their own parameters for time, etc.) and get a report.

[taiki-e]

cc @smoelius: Do you have any recommendations or opinions on the interface or behavior here? I guess you are the best person to ask about this because you are a cargo-afl maintainer and cargo-llvm-cov contributor:)

[smoelius]

cc @smoelius: Do you have any recommendations or opinions on the interface or behavior here? I guess you are the best person to ask about this because you are a cargo-afl maintainer and cargo-llvm-cov contributor:)

It's very kind of you to ask. :)

I think your proposed interface makes a lot of sense.

One thing I note about the current PR is that it contains code one would expect to find in cargo-afl, e.g., docs/cargo-llvm-cov-afl.txt.

FWIW, cargo afl subcommand will run cargo subcommand with AFL-instrumented binaries. I'm not sure if this makes your task easier, @xizheyin.

As an aside, someone recently asked for coverage on the afl.rs repo: https://github.com/rust-fuzz/afl.rs/issues/465

[taiki-e]

@xizheyin

By the way, how do I get the coverage of a library?

Do you mean "coverage of dependencies (outside the workspace)"? In cargo-llvm-cov, they are excluded by default, but should show up if you pass the --disable-default-ignore-filename-regex flag.

[xizheyin]

@taiki-e Yes, it's what it means. This worked, but this lists the coverage of all files as shown below. Is it possible to count the overall coverage of this dependency library?

[taiki-e]

Is it possible to count the overall coverage of this dependency library?

Opened https://github.com/taiki-e/cargo-llvm-cov/pull/353 to implement this.

[xizheyin]

@taiki-e Sorry to interrupt again. If I have 10 different programs that all use a certain library and I have access to their profraws, is it possible to easily count the total coverage of this library by these ten programs?

[taiki-e]

Sorry to interrupt again. If I have 10 different programs that all use a certain library and I have access to their profraws, is it possible to easily count the total coverage of this library by these ten programs?

If it is possible to (temporarily) combine the crates to be tested into a single workspace to run tests, perhaps it would be easy to do by combining with --dep-coverage.

If not, it would be difficult with the current implementation since only artifacts related to the current workspace would be collected, but I think that could be handled by adjusting object_files (probably just making this if branch unconditional -- of course that is not a reasonable default behavior although), specifying the same target directory by CARGO_TARGET_DIR, and using --no-report + report pattern.

In any case, could you please open a separate issue for questions not directly related to this PR?

[xizheyin]

Thanks, I will open a issue if I have any question.

[njelich]

It seems sufficient to just add an "external tests" guide, but for AFL - this seems to work with the tutorial fuzzer in the AFL docs:

source <(cargo llvm-cov show-env --export-prefix)
cargo llvm-cov clean --workspace
cargo afl build
AFL_FUZZER_LOOPCOUNT=20 cargo afl fuzz -c - -V 10 -i in -o out target/debug/url-fuzz-target
cargo llvm-cov report --html

And it allows full customization without overhauling the enitrety of the llvm-cov args system.