dvyukov / go-fuzz

Randomized testing for Go
Apache License 2.0
4.75k stars 276 forks source link

pass fuzz.F to fuzz functions #218

Open josharian opened 5 years ago

josharian commented 5 years ago

The current Fuzz function signature is

func FuzzSomething(b []byte) int

I think we should migrate it to something more like:

import fuzz "github.com/dvyukov/go-fuzz"

func FuzzSomething(fz fuzz.F)

(That import path will obviously have to change if go-fuzz moves into the standard toolchain. Or if we migrate to github.com/go-fuzz/go-fuzz, or the like.)

I imagine starting fuzz.F (fuzzing.F?) with:

type F interface {
  // Bytes returns a byte slice to be used as input.
  Bytes() []byte

  // Skip tells go-fuzz that this input should not added to the corpus, and stops execution.
  // (Similar to return -1 right now.)
  // msg explains why. It is currently just a form of documentation for the user,
  // but you can imagine later gathering stats about whence all the skips.
  Skip(msg string)

  // Interesting tells go-fuzz that this input is interesting and should be given added priority.
  // Equivalent to return 1 right now, except that it does not stop execution.
  Interesting()

  // Fail reports a failure of an invariant and stops execution.
  // Equivalent to a custom panic right now.
  Fail(msg string)
  Failf(msg string, ...interface{})

  // ExitOnCompletion requests that go-fuzz exit the binary after the Fuzz function completes.
  // This dramatically impacts performance and the effectiveness of go-fuzz;
  // it should be used only when it is infeasible to write a fuzz function that can be safely
  // called multiple times.
  ExitOnCompletion()
}

There's plenty more to add, e.g. key-value-based requests for bools/ints/etc instead of having to parse them out of a byte slice. But this would be a good first start.

In order to avoid people having to change their fuzz functions, I'd automatically detect the old style of signature and have go-fuzz-build insert a shim.

Discuss. :)

(P.S. I think you had a similar proposal, Dmitry. I know that I need to go look at it. Apologies.)

hrissan commented 5 months ago

Another method to consider is AddDataCoverage(hash uint64)

We are experimented with data space-guided coverage and it shows very strong result. Basically if we hash our model state and modify coverage bitmap based on this hash, then fuzzer is able to explore not only code space, but data space as well.

Explained here.

https://ieeexplore.ieee.org/document/9152719