Open tevoinea opened 1 year ago
I was able to write up a small rust binary that takes the path to a Go project with a fuzz test written and fuzzes it while extracting the following runtime information: execs/s, new interesting inputs, total interesting inputs, and crashing/failing inputs.
However, gofuzz doesn't yet support continuous fuzzing and will stop fuzz iterations after finding the first failing input. A flag exists to change this behavior but is not part of any releases. It looks like the Go team is planning to include this feature in one of the upcoming releases, but for now there doesn't seem to be a way around this.
This issue in addition to the coverage problem Teo faced are currently blocking any further progress on integrating the native Go fuzzer into OneFuzz.
@tevoinea Regaring "Code coverage from fuzzing execution" requirement. There is an option: build fuzz target and test it separately.
Build with -cover
and -o
options:
go test . -fuzz=FuzzSomething -o fuzz -c -cover
(need to specify -fuzz
once with any available FuzzXxx function even if we have few of them)
This will create separate fuzz
binary.
To fuzz run this separate binary with -test.fuzz
and -test.coverprofile
options:
./fuzz -test.fuzz=FuzzSomething -test.coverprofile=coverprofile_FuzzSomething.out
(can specify other available functions in -fuzz
if any, no rebuild required)
Generate coverage report as usual:
go tool cover -html=coverprofile_FuzzSomething.out -o=coverage_FuzzSomething.html
UPD: currently this doesn't work (returns 0% coverage). What a bummer...
@fuzzah Thanks, that's great! We put this issue on pause (on our end) for the time being but when we revisit it, I think that means we have everything we need to get the feature over the line.
Turns out the support for using both -test.fuzz
and -test.coverprofile
seems to be a bug..........
When running fuzz tests without building we can't use -fuzz
and -coverprofile
, as it gave 0% coverage and was temporarily disabled with error message ( https://github.com/golang/go/issues/48178#issuecomment-920216744 )
Building and then using flags -test.fuzz
and -test.coverprofile
"works" but also gives 0% coverage.
So currently there seems to be no way to fuzz and collect coverage at the same time. Sorry for the false alarm! :(
Another thing is that the stacktrace parser needs updating to handle Go stacktraces in the same way that #2988 did for .NET stacktraces. (Added this to the first post.)
go
supports fuzzing natively in the cli tool. In order to support this method of fuzzing in OneFuzz we need:Produce more than 1 crashing input when running the fuzzer
Elaborated here: https://github.com/microsoft/onefuzz/issues/2429#issuecomment-1253847924
Reproduceable crashes
This is possible today. When the fuzzer finds a crash, a file is stored with the inputs that caused the crash. The inputs are 'shareable' across machines.
Code coverage from fuzzing execution
Some coverage is available as a summary of the total number of lines covered however line level detail to produce coverage maps like cobertura is not available for fuzzing. I've started a discussion here: https://forum.golangbridge.org/t/code-coverage-from-fuzzing/28875
AB#36056