Fuzzing stops after certain time

[prksastry]

I am getting the following error message after 30 minutes of fuzzing. [!] WARNING: Error delivering sample, retrying with a clean target [-] PROGRAM ABORT : Repeatedly failed to deliver sample Location : Fuzzer::RunSampleAndGetCoverage(), C:\Users\cdac\Documents\Jackalope-main\fuzzer.cpp:216 May I know the reasons for program abort?

[ifratric]

Hmm I heard from other people encountering the same error, but I wasn't able to reproduce myself. Can you reproduce with the test target provided with jackalope (when running using file delivery mechanism, e.g. fuzzer.exe -in in -out out -t 1000 -delivery file -instrument_module test.exe -target_module test.exe -target_method fuzz -nargs 1 -iterations 10000 -persist -loop -cmp_coverage -- test.exe -f @@ ) or only with your own target?

In general, this can happen if Jackalope can't write a sample to the output file. In case the target still holds a reference to the file and that's the reason the file can't be written, Jackalope attempts to kill the target process and retry as can be seen in https://github.com/googleprojectzero/Jackalope/blob/main/fuzzer.cpp#L214. But in your case, it appears that that didn't resolve the issue.

Currently, the best workaround is to use sample delivery via shared memory (see test.cpp for an example on how to modify your target to support it). This will likely also result in the increased fuzzing speed.

[prksastry]

Hmm I heard from other people encountering the same error, but I wasn't able to reproduce myself. Can you reproduce with the test target provided with jackalope (when running using file delivery mechanism, e.g. fuzzer.exe -in in -out out -t 1000 -delivery file -instrument_module test.exe -target_module test.exe -target_method fuzz -nargs 1 -iterations 10000 -persist -loop -cmp_coverage -- test.exe -f @@) or only with your own target?

In general, this can happen if Jackalope can't write a sample to the output file. In case the target still holds a reference to the file and that's the reason the file can't be written, Jackalope attempts to kill the target process and retry as can be seen in https://github.com/googleprojectzero/Jackalope/blob/main/fuzzer.cpp#L214. But in your case, it appears that that didn't resolve the issue.

Currently, the best workaround is to use sample delivery via shared memory (see test.cpp for an example on how to modify your target to support it). This will likely also result in the increased fuzzing speed.

when the delivery is via shared memory, the speed is observed to be very high, there are some instances when the APIs require only file or file handles, yesterday I have read the blog https://www.zerodayinitiative.com/blog/2019/10/31/the-little-bitmap-that-couldnt where they have implemented a minimal harness application, I copied the same harness and started fuzzing it via Jackalope with flag "-delivery shmem" , I expected the fuzzer to throw an error since shared memory is not implemented in the application, but instead it started fuzzing with multiples of thousands execs/sec, can I know why I have not encountered error in this case, while in previous case when i try to fuzz custom WinAFL harness, it allowed only "-delivery file" ??

[ifratric]

Most likely the target is doing something and you're getting coverage for that, but that "something" isn't related to the input sample if, as you say, you're passing it over shared memory and the target is not aware of this shared memory.

[5n1p3r0010]

Hmm I heard from other people encountering the same error, but I wasn't able to reproduce myself. Can you reproduce with the test target provided with jackalope (when running using file delivery mechanism, e.g. fuzzer.exe -in in -out out -t 1000 -delivery file -instrument_module test.exe -target_module test.exe -target_method fuzz -nargs 1 -iterations 10000 -persist -loop -cmp_coverage -- test.exe -f @@) or only with your own target?

In general, this can happen if Jackalope can't write a sample to the output file. In case the target still holds a reference to the file and that's the reason the file can't be written, Jackalope attempts to kill the target process and retry as can be seen in https://github.com/googleprojectzero/Jackalope/blob/main/fuzzer.cpp#L214. But in your case, it appears that that didn't resolve the issue.

Currently, the best workaround is to use sample delivery via shared memory (see test.cpp for an example on how to modify your target to support it). This will likely also result in the increased fuzzing speed.

Hi,fratric,I get the same issue when fuzzing some windows codecs using jackalope,with delivery=file and nthreads=10.Cmd line like:

fuzzer.exe -delivery file -nthreads 10 -t 1000 -in D:\corpus\xx -out R:\Temp\xx -instrument_module xx.dll -target_module fuzzer.exe -target_method fuzz -nargs 1 -iterations 2337 -persist -loop -cmp_coverage -- fuzzer.exe @@

Without nthreads=10 option,seems jackalope works fine with the same harness code.

[ifratric]

Thank you for letting me know this is related to nthreads. It's interesting because each thread is writing to it's own file so there shouldn't be race conditions there (but I'll doublecheck it just in case).

In the meantime, can you test if putting

Sleep(10000);

before or after https://github.com/googleprojectzero/Jackalope/blob/364bf09f0821182f330c2b3ebe6ce840e23dde51/fuzzer.cpp#L213 helps with the issue at all?

[ifratric]

I managed to reproduce the issue. It should be fixed in the latest version, commit https://github.com/googleprojectzero/Jackalope/commit/2f72da12bc7c363770712db26c99fe490b68779d though note some warnings will still be printed.

Tentatively closing. Please reopen if the same issue still persist even after the patch.