search

maflcko / b-c-cov

Bitcoin Core coverage report
https://maflcko.github.io/b-c-cov/
12 stars 3 forks source link

Issue Reproducing fuzzing Coverage #5

Closed RobinDavid closed 2 days ago

RobinDavid commented 4 days ago

Dear @maflcko,

While trying to reproduce the fuzzing coverage generated by you CI, I encountered two issues.

  1. I had to add --base-directory ../src as parameter to lcov command for it to find source files. It seems your CI is having the same problem at the moment: https://github.com/maflcko/b-c-cov/runs/33454998185

  2. After fixing the cmake issue I can't manage to obtain the same coverage than yours here: https://maflcko.github.io/b-c-cov/fuzz.coverage/index.html

Mine looks like this: qa-assets_screenshot

I am focusing on the fuzzing coverage. Thus I am running the same commands than one in "fuzz_lcov" in your cirrus.yml. I am rather confident it runs correctly on qa-assets corpus.

Do you have any ideas ? (Maybe the two issues are linked).

Regards, Robin

maflcko commented 4 days ago

You may want to try with https://github.com/bitcoin/bitcoin/pull/31337 ?

RobinDavid commented 4 days ago

Nice! I looked bitcoin opened issues, but I apparently missed it. I will try.

RobinDavid commented 4 days ago

It correctly runs with that branch! Still, the coverage is still very low in comparison to what you obtain (2.).

I run the same command with qa-assets at the same location: cmake -DJOBS=$NUM_MAKE -DFUZZ_CORPORA_DIR=$PWD/qa-assets/fuzz_corpora -DLCOV_OPTS='--rc branch_coverage=1 --ignore-errors mismatch,mismatch,inconsistent,inconsistent' -P ./bld-fuzz/CoverageFuzz.cmake

Do you have any ideas ?

maflcko commented 4 days ago

Hmm, hard to say. Possibly there could be leftover files from a previous run?

I usually spin up a new container to run the coverage stuff. Can you reproduce in a fresh container as well? If yes, it would be good to have the exact steps to reproduce. (Otherwise, can you reproduce with a fresh clone of all git repos?)

RobinDavid commented 2 days ago

I did not managed to backtrack the exact origin of the issue, but the merging of branch https://github.com/bitcoin/bitcoin/pull/31337 fixed it. I am closing the issue.