smartystreets / goconvey

Go testing in the browser. Integrates with `go test`. Write behavioral tests in Go.
http://smartystreets.github.io/goconvey/
Other
8.23k stars 554 forks source link

Support Go 1.18 fuzz feature #659

Open srabraham opened 2 years ago

srabraham commented 2 years ago

There's now a fuzzing feature to Go testing: https://go.dev/doc/fuzz/ It's a bit different than benchmarking, since these are real tests that have assertions in them. GoConvey seems like a good framework to consider.

It's not currently pretty using this feature with GoConvey though. Beyond any other UX questions, the most obvious thing is that you don't start with a *testing.T, but rather a *testing.F. That means that the top-level Convey call needs to be done inside the f.Fuzz, which is kind of weird. Anyway, properly supporting fuzzing everywhere in GoConvey would probably take some effort, but it's worth considering.

func FuzzSomethingSilly(f *testing.F) {
    f.Add(0)
    f.Add(1)
    f.Add(2)
    f.Fuzz(func(t *testing.T, n uint64) {
        Convey("For fuzz input", t, func() {
            So(n, ShouldEqual, n)
        })
    })
}
srabraham commented 2 years ago

@riannucci are you pretty much the owner of this project now? It's good to see you again! 😄

riannucci commented 2 years ago

Lol yep, I'm kind of the de-facto maintainer at this point... I'm unfortunately not able to give the project the attention it deserves :( (if you wanna pitch in, please do so!)

Yeah I wonder if we can use testing.TB instead of *testing.T in the convey interfaces? That might be sufficient?

riannucci commented 2 years ago

(and yeah there's a whole bunch of UX questions... I feel like Convey is due for a rework on how it internally tracks test state and how it does stdio, too, in order to take advantage of "new" features like testing.Run which was added in Go1.7.....)