CodeIntelligenceTesting / jazzer

Coverage-guided, in-process fuzzing for the JVM
https://code-intelligence.com
Other
1.01k stars 135 forks source link

[Q] How to enable value profiling when running a Junit @FuzzTest ? #668

Open mdindoffer opened 1 year ago

mdindoffer commented 1 year ago

The @FuzzTest annotation has only one config parameter maxDuration. How can I pass the -use_value_profile=1 to libFuzzer when running a Junit fuzzTest from IntelliJ IDE? I naively tried adding this as an environment property (thinking this could be handled the same way as JAZZER_FUZZ), and a java system property, but that didn't help.

Do I have to use the standalone jazzer binary to use other libFuzzer options? The "Advanced techniques" docs only tell me to pass this flag, but since it's the JVM starting libfuzzer indirectly, I don't know how to do this.

mdindoffer commented 1 year ago

OK, I think I found the answer browsing the code at https://github.com/CodeIntelligenceTesting/jazzer/blob/cec3a817971f75595e23a729b2af81d61e0cf21c/src/main/java/com/code_intelligence/jazzer/junit/FuzzTestExecutor.java#L140

The solution is to add a system property with the name hardcoded in jazzer before executing the unit test.

    @BeforeAll
    static void beforeAll() {
        System.setProperty("jazzer.valueprofile", "true");
    }

I will leave this open for someone to confirm that this is actually the intended way to do this. Might be worth adding to the docs.

ghost commented 6 months ago

Hi @mdindoffer - I guess that we can neither confirm nor deny....lol I'm going through issues and checking in. Are you still using Jazzer? A lot has changed in Jazzer since March 2023. Ping me to discuss. david[dot]merian [at] code-intelligence[dot]com

mdindoffer commented 6 months ago

Ah I forgot about this. I am using Jazzer from time to time, haven't used it since October 2023 though (version 0.21.1).

~That said, my previous comment was wrong. Setting the value via System.setProperty makes no difference, because that sets only the Java "System Property", while Jazzer is (was?) looking for environment variables.~

~Therefore, I ended up setting the environment variable textfield in my Intellij Idea IDE run configurations with JAZZER_FUZZ=1;jazzer.valueprofile=true.~ image

~This works for me well enough, since I'm used to fuzz from the IDE anyway. For CLI/CI one has to pass the value in a different way of course.~

Well, to further confuse you, I was wrong. I just checked and seems the System.setProperty call is actually the way to go, lol.