rohanpadhye / JQF

JQF + Zest: Coverage-guided semantic fuzzing for Java.
BSD 2-Clause "Simplified" License
647 stars 109 forks source link

Maven plugin gets stack overflow error #251

Open reftel opened 2 weeks ago

reftel commented 2 weeks ago

I´m writing a simple fuzz runner. To isolate the interesting parts of the application, the test uses some Mockito mocks. Starting the runner via maven using "jqf:fuzz" fails with the following stack trace (abbreviated):

Exception in thread "Logging-Cleaner" java.lang.StackOverflowError
    at edu.berkeley.cs.jqf.instrument.util.DoublyLinkedList.remove(DoublyLinkedList.java:140)
    at edu.berkeley.cs.jqf.instrument.util.DoublyLinkedList.synchronizedRemove(DoublyLinkedList.java:220)
    at edu.berkeley.cs.jqf.instrument.tracing.SingleSnoop$1.initialValue(SingleSnoop.java:52)
    at edu.berkeley.cs.jqf.instrument.tracing.SingleSnoop$1.initialValue(SingleSnoop.java:48)
    at java.base/java.lang.ThreadLocal.setInitialValue(ThreadLocal.java:195)
    at java.base/java.lang.ThreadLocal.get(ThreadLocal.java:172)
    at edu.berkeley.cs.jqf.instrument.tracing.SingleSnoop.METHOD_BEGIN(SingleSnoop.java:1002)
    at org.mockito.internal.creation.bytebuddy.MockMethodAdvice.isMock(MockMethodAdvice.java)
    at java.base/java.lang.Object.equals(Object.java:163)
    at edu.berkeley.cs.jqf.instrument.util.DoublyLinkedList.remove(DoublyLinkedList.java:140)
    at edu.berkeley.cs.jqf.instrument.util.DoublyLinkedList.synchronizedRemove(DoublyLinkedList.java:220)
    at edu.berkeley.cs.jqf.instrument.tracing.SingleSnoop$1.initialValue(SingleSnoop.java:52)
    at edu.berkeley.cs.jqf.instrument.tracing.SingleSnoop$1.initialValue(SingleSnoop.java:48)
    at java.base/java.lang.ThreadLocal.setInitialValue(ThreadLocal.java:195)
    at java.base/java.lang.ThreadLocal.get(ThreadLocal.java:172)
    at edu.berkeley.cs.jqf.instrument.tracing.SingleSnoop.METHOD_BEGIN(SingleSnoop.java:1002)
    at org.mockito.internal.creation.bytebuddy.MockMethodAdvice.isMock(MockMethodAdvice.java)

...

    at org.mockito.internal.creation.bytebuddy.MockMethodAdvice.isMock(MockMethodAdvice.java)
    at java.base/java.lang.Object.equals(Object.java:163)
    at edu.berkeley.cs.jqf.instrument.util.DoublyLinkedList.remove(DoublyLinkedList.java:140)
    at edu.berkeley.cs.jqf.instrument.util.DoublyLinkedList.synchronizedRemove(DoublyLinkedList.java:220)
    at edu.berkeley.cs.jqf.instrument.tracing.SingleSnoop$1.initialValue(SingleSnoop.java:52)
    at edu.berkeley.cs.jqf.instrument.tracing.SingleSnoop$1.initialValue(SingleSnoop.java:48)
    at java.base/java.lang.ThreadLocal.setInitialValue(ThreadLocal.java:195)
    at java.base/java.lang.ThreadLocal.get(ThreadLocal.java:172)
    at edu.berkeley.cs.jqf.instrument.tracing.SingleSnoop.METHOD_BEGIN(SingleSnoop.java:1002)
    at org.mockito.internal.creation.bytebuddy.MockMethodAdvice.isMock(MockMethodAdvice.java)
    at java.base/java.lang.Object.equals(Object.java:163)
    at edu.berkeley.cs.jqf.instrument.util.DoublyLinkedList.remove(DoublyLinkedList.java:140)
    at edu.berkeley.cs.jqf.instrument.util.DoublyLinkedList.synchronizedRemove(DoublyLinkedList.java:220)
    at edu.berkeley.cs.jqf.instrument.tracing.SingleSnoop$1.initialValue(SingleSnoop.java:52)
    at edu.berkeley.cs.jqf.instrument.tracing.SingleSnoop$1.initialValue(SingleSnoop.java:48)
    at java.base/java.lang.ThreadLocal.setInitialValue(ThreadLocal.java:195)
    at java.base/java.lang.ThreadLocal.get(ThreadLocal.java:172)
    at edu.berkeley.cs.jqf.instrument.tracing.SingleSnoop.METHOD_BEGIN(SingleSnoop.java:1002)
rohanpadhye commented 2 weeks ago

It looks like Mockito is overriding Thread.equal() and so instrumented code (in this case Mockito) is running when JQF is trying to perform it's core logic involving threads, causing infinite recursion.

Can you try excluding org.mockito from the instrumentation via the -Dexcludes flag?

reftel commented 2 weeks ago

With -Dexcludes=org/mockito, that error went away. Thanks! Since this is really dependent on the test (some tests require it, some don´t), would it make sense to have excludes as a parameter in the Fuzz annotation? Also, it seems to me that it would make using JQF easier for beginners if there were a default exclude list for known problematic libraries. Do you agree?

rohanpadhye commented 2 weeks ago

Thanks for the feedback.

We have to set excludes at the command-line level because JQF needs to know whether or not to perform instrumentation during class-loading. If a class is loaded with instrumentation, it cannot be undone (at least not in a straightforward way).

I would imagine that if you want to exclude a library from instrumentation because it is not providing high-level logic but only providing some language-level utilities (e.g., Mockito) then you would want to exclude it permanently.

I like the idea of providing more defaults for problematic libraries. We do have something like this for the command-line driver but not the Maven plugin (mainly because these banned libraries are already pre-loaded by the system class loader when the Maven plugin kicks in).