CodeIntelligenceTesting / jazzer

Coverage-guided, in-process fuzzing for the JVM
https://code-intelligence.com
Other
1.03k stars 137 forks source link

How can I ensure that the fuzzing process continues even in the event of a crash? #770

Closed wzzll123 closed 1 year ago

wzzll123 commented 1 year ago

Every time Jazzer encounters a crash, it will halt and generate a corresponding reproducer. However, I desire the fuzzer to continue running for more interesting results instead of immediately ceasing. Is it possible to achieve this, perhaps by adding additional options?

br-lewis commented 1 year ago

I think this option is what you want: https://github.com/CodeIntelligenceTesting/jazzer/blob/main/src/main/java/com/code_intelligence/jazzer/driver/Opt.java#L117-L118

wzzll123 commented 1 year ago

Thank you very much for your valuable reply. This option looks helpful, however, when I tried it in my command: ../../common_utilities/jazzer --cp=$JAR_PATH --jvm_args="-Xmx2048m:-Xss1024k" --target_class=$fuzzer --keep_going=20 -timeout=2 corpus

The jazzer still stops when encounter the first timeout: -ChangeBit-Custom- DE: "D\000\000\000"-

386670 REDUCE cov: 2604 ft: 13108 corp: 1105/348Kb lim: 4096 exec/s: 589 rss: 1019Mb L: 292/4096 MS: 2 EraseBytes-Custom-

392451 REDUCE cov: 2604 ft: 13108 corp: 1105/348Kb lim: 4096 exec/s: 589 rss: 1019Mb L: 296/4096 MS: 2 EraseBytes-Custom-

ALARM: working on the last Unit for 3 seconds and the timeout value is 2 (use -timeout=N to change) MS: 6 ChangeByte-Custom-ChangeBit-Custom-CopyPart-Custom-; base unit: f5b12ad34bab25d18b750d587eb17a4a76526828 artifact_prefix='./'; Test unit written to ./timeout-190a07de5858e0787b9ee930a7773248bd12fae8 ==79074== ERROR: libFuzzer: timeout after 3 seconds

Stack traces of all JVM threads: Thread[Common-Cleaner,8,InnocuousThreadGroup] at java.base@11.0.12/java.lang.Object.wait(Native Method) at java.base@11.0.12/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155) at java.base@11.0.12/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148) at java.base@11.0.12/java.lang.Thread.run(Thread.java:834) at java.base@11.0.12/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:134)

Thread[Finalizer,8,system] at java.base@11.0.12/java.lang.Object.wait(Native Method) at java.base@11.0.12/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155) at java.base@11.0.12/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176) at java.base@11.0.12/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:170)

Thread[Attach Listener,9,system]

Thread[main,5,main] at app//com.code_intelligence.jazzer.driver.FuzzTargetRunner.dumpAllStackTraces(FuzzTargetRunner.java:487)

Thread[Reference Handler,10,system] at java.base@11.0.12/java.lang.ref.Reference.waitForReferencePendingList(Native Method) at java.base@11.0.12/java.lang.ref.Reference.processPendingReferences(Reference.java:241) at java.base@11.0.12/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:213)

Thread[Signal Dispatcher,9,system]

Garbage collector stats:

PS MarkSweep: 0 collections took 0ms PS Scavenge: 1536 collections took 7644ms

SUMMARY: libFuzzer: timeout

bertschneider commented 1 year ago

That setting applies to errors found during the execution of your Java code, so uncaught exceptions or bug detector findings. On the other hand, timeouts are handled by the internally used libFuzzer instance, which doesn't honor it.

We actually discussed this problem a few days back and didn't see a practical way to skip timeouts. As you also ran into it, it's probably a good idea to reinvestigate possible solutions.

In the meantime, why are the timeouts generated in the first place? Couldn't you change your code to prevent them altogether or use an appropriate/higher value for the functionality you want to fuzz?

fmeum commented 1 year ago

libFuzzer's -fork continues fuzzing past timeouts by default. Please give that a try and reopen if needed.