Closed lukacan closed 3 months ago
Yes, so I updated the feature implementation. Essentially:
stdout
of the cargo hfuzz
command is marked as piped and is parsed within Trident after the fuzzing session.HashMap
and then serialized into JSON format.
HashMap
(line by line).HashMap
is displayed (as table), containing the statistics.Points to note:
cargo hfuzz
stdout
is piped, not stderr
. This is because redirecting stderr
would also redirect the compilation output, which is undesirable.@Ikrk Ready for review
I added also stats of failed invariants checks and I created a new Jira task to potentially extend the stats also with unhandled panics: https://ackeeblockchain.atlassian.net/browse/TRD-81
Currently, it is not possible to determine (or somehow debug) whether the fuzzing session is actually fuzzing something and if the instructions are successfully executed. This means that based solely on the fuzzer's output and zero crashes, the user cannot tell if their program has no errors or if none of the targeted instructions were successfully executed.
This PR introduces a simple stats logging mechanism.
The best option would be to show stats for the fuzzing session right after the session ends. However, the honggfuzz workflow is as follows:
Due to the
SIGKILL
, I think it is not possible to time the end of a fuzzing subprocess (fuzzing session) = we do not exactly know when to output the stats.So, this PR aims to provide stats during the whole fuzzing session. A new function called
run_fuzzer_with_stats
inside the commander is created.The stats collect an accumulating number of invocations and successful invocations of corresponding instructions. Then, at the end of a fuzzing iteration, stats are printed. If every instruction was successfully executed, the whole structure with a success message is printed; on the other hand, if any instruction has 0 successful invocations, an error message is printed.
This means as the underlying fuzzer tries to explore as many branches as possible, this can lead to two scenarios:
Lastly, we have two options for indicating to the fuzzer that we want to see stats.
I find the second option better because it has no impact on performance and for the first option, we can use:
However, this approach results in a full re-compilation even if
trident fuzz run <FUZZ_TARGET>
was already called.NOTE: regarding the
text_generator.rs
changes. Thecargo clippy
had problem with (used) unused variable SUCCESS within thelib.rs
. So I just changed FINISH -> SUCCESS.