google / gofuzz

Fuzz testing for go.
Apache License 2.0
1.5k stars 119 forks source link

Generating blank strings even though nilChance is set to 0 #45

Closed gmgchow closed 4 years ago

gmgchow commented 4 years ago

I am trying to fuzz a string field like below, but it is returning empty string ("") even though I set the nilChance to zero. The field I am fuzzing does not support empty strings so I would like to exclude it from my fuzzing input. Is there a way to do this?

for i := 0; i < 100; i++ {
  var s string
  fuzz.New().NilChance(0).Fuzz(&s)
  _, err := MyMethod(s) // this function will return an error if s is an empty string
  if err != nil {
    // handle error
  }
}
lavalamp commented 4 years ago

Yes, NilChance doesn't affect string length.

You should make a custom fuzz func, either for string or for the structure containing the string. e.g.,

f := fuzz.New().Funcs(
  func(o *MyStruct, c fuzz.Continue) {
    c.FuzzNoCustom(o) // Make an ordinary fuzz pass without calling the custom fuzz functions
    for o.MyString == "" { c.Fuzz(&o.MyString) }
  })

Chances are, if your string must not be empty, there's some other qualities you may want to enforce on it as well.

It's often valuable to run your code over invalid inputs, depending on your situation, to make sure it doesn't do something really bad. Custom fuzz functions in my view are mostly about adjusting the probability of invalid input. E.g., if you've got 20 strings with this requirement, then nearly all of the time at least one of them will be blank, and the fuzzer will almost never test your success path.

gmgchow commented 4 years ago

I see, thank you for the suggestions and code sample! I will close this issue since it was a misunderstanding on my part.