search

flyingmutant / rapid

Rapid is a modern Go property-based testing library
https://pkg.go.dev/pgregory.net/rapid
Mozilla Public License 2.0
579 stars 25 forks source link

Error using built in generators - group did not use any data from bitstream #65

Closed peterargue closed 7 months ago

peterargue commented 7 months ago

I'm running v1.1.0

I get the following error randomly running tests that exclusively use of the built in generators:

panic: group did not use any data from bitstream; this is likely a result of Custom generator not calling any of the built-in generators

goroutine 554801 [running]:
pgregory.net/rapid.assertf(...)
        /Users/asdf/go/pkg/mod/pgregory.net/rapid@v1.1.0/engine.go:81
pgregory.net/rapid.(*recordedBits).endGroup(0x30?, 0x100580000?, 0xe8?)
        /Users/asdf/go/pkg/mod/pgregory.net/rapid@v1.1.0/data.go:125 +0x10c
pgregory.net/rapid.genUintNBiased({0x1062003b0, 0x14000cd1080}, 0x26ac)
        /Users/asdf/go/pkg/mod/pgregory.net/rapid@v1.1.0/utils.go:69 +0xc0
pgregory.net/rapid.genUintN({0x1062003b0?, 0x14000cd1080?}, 0x8?, 0x7?)
        /Users/asdf/go/pkg/mod/pgregory.net/rapid@v1.1.0/utils.go:93 +0x28
pgregory.net/rapid.genUintRange({0x1062003b0?, 0x14000cd1080?}, 0x64, 0x140006d7980?, 0xb8?)
        /Users/asdf/go/pkg/mod/pgregory.net/rapid@v1.1.0/utils.go:104 +0x3c
pgregory.net/rapid.(*integerGen[...]).value(0x1060bada0?, 0x14000d537e8?)
        /Users/asdf/go/pkg/mod/pgregory.net/rapid@v1.1.0/integers.go:269 +0x64
pgregory.net/rapid.(*Generator[...]).value(0x0, 0x14000f0c200?)
        /Users/asdf/go/pkg/mod/pgregory.net/rapid@v1.1.0/generator.go:74 +0x64
pgregory.net/rapid.(*Generator[...]).Draw(0x106210160?, 0x14000f0c200, {0x105bd3157, 0xa})
        /Users/asdf/go/pkg/mod/pgregory.net/rapid@v1.1.0/generator.go:47 +0x60
github.com/myproject/engine/access/rpc/connection.TestConcurrentConnectDisconnect.func2.1({0x106d0e2a0?, 0x140005c1f40?}, 0x2?)
        /Users/asdf/go/src/github.com/myproject/engine/access/rpc/connection/connection_test.go:744 +0x70

The specific line it fails on is

sleepMicro := rapid.UintRange(100, 10_000).Draw(tt, "sleepMicro")

The test involves some testify mocks and a grpc client/server interaction. It fails with this panic ~1/20 runs.

Unfortunately, I haven't been able to create a reproduction that doesn't involve a bunch of our test code, but I have seen it in a couple different tests.

flyingmutant commented 7 months ago

That's really strange, the panic you are seeing comes from endGroup call at line 69

https://github.com/flyingmutant/rapid/blob/d56e0371c298310defbd867b24ab21ef7bf169bb/utils.go#L64-L69

However, that group always contain a call to genGeom, which always draws 53 bits:

https://github.com/flyingmutant/rapid/blob/d56e0371c298310defbd867b24ab21ef7bf169bb/utils.go#L26-L37

Maybe there is something incorrect happening with regards to concurrent access to generators? Note the documentation:

For tests to be reproducible, they should generally run in a single goroutine. If concurrency is unavoidable, methods on T, such as [testing.T.Helper](https://pkg.go.dev/testing#T.Helper) and *T.Errorf, are safe for concurrent calls, but Generator.Draw from a given T is not.

peterargue commented 7 months ago

Thanks for getting back so quick @flyingmutant!

Yes, it seems to be an issue with concurrency. I wrapped my call to Draw with a mutex and the crashes stopped. I misinterpreted that note to mean for a given generator, not T. Thanks for clarifying.