Code coverage instrumentation crashes without a stack trace.

[svenkeidel]

Hi, thanks for the great work!

I tried instrumenting all classes in the JDK with --instrumentation_includes=java.**. However, during instrumentation jazzer crashes without a stack trace:

...
IINFO: Instrumented java.lang.CharSequence (took 1 ms, size +17%)
INFO: Instrumented java.lang.Comparable (took 0 ms, size +0%)
INFO: Instrumented java.io.Serializable (took 0 ms, size +0%)
Exception occurred in thread "main"

This makes it hard to understand what is going wrong.

To reproduce run

git clone -b instrument-jdk https://github.com/svenkeidel/example-fuzzing
cd example-fuzzing
./fuzzing.sh

[bertschneider]

Thanks for the report! We will take a look.

[fmeum]

I tested this on some of Jazzer's own examples and ran into a different exception when trying to instrument java.**:

Exception: java.lang.ClassCircularityError thrown from the UncaughtExceptionHandler in thread "main"

The error went away when I added java.util.** to --instrumentation_excludes. Without patching the JDK to get more logs it's going to be hard to determine the particular classes that cause this circularity. Jazzer's instrumentation engine heavily relies on java.util.**, so we might not be able to support this use case easily.

What are you ultimately trying to achieve? Since broad instrumentation filters, even when they don't lead to internal crashes, usually hurt the performance of the fuzzer, could you try using a more targeted filter?

[svenkeidel]

I work in a research group at TU Darmstadt (https://www.stg.tu-darmstadt.de/). We use Jazzer to dynamically record a call graph, which we compare to a static call graph to measure precision and recall. To assess the quality of the dynamic call graph, we would like to understand how well the dynamic call graph covers the complete program, including the JDK.

That said, we now record coverage in a subsequent run without Jazzer. The subsequent run executes the program with inputs from the corpus jazzer created, while Jacoco records coverage and our own JVMTI agent records calls.

So from my side, the issue can be closed. That said, it would have been helpful if Jazzer could list the classes that could not be instrumented, rather than crashing with a non-descriptive error message.

Thanks for investigating this.

[fmeum]

Thanks for providing the context, this is very interesting.

So from my side, the issue can be closed. That said, it would have been helpful if Jazzer could list the classes that could not be instrumented, rather than crashing with a non-descriptive error message.

It generally does this and also includes the exception that is the reason for a class not having been instrumented. However, determining these classes and printing the exceptions requires List, String and some other classes from java.util and java.lang, so if those fail to be instrumented for some reason, I fear there is no easy way to print the failure from Java.