flyingmutant / rapid

Rapid is a modern Go property-based testing library
https://pkg.go.dev/pgregory.net/rapid
Mozilla Public License 2.0
579 stars 25 forks source link

A way to pass custom data to `StateMachine` instances #48

Closed danwt closed 1 year ago

danwt commented 1 year ago

I am depending on libraries that require a concrete testing.T, and I want to use these libraries in the Init method of a state machine test. Is it possible?

The signature of Init takes a * rapid.T which isn't useable

https://github.com/flyingmutant/rapid/blob/24f65e311277c10f9de268aa2e0c68287e09c74c/statemachine.go#L108-L118

type Model struct {
    // .. stuff
}

func (m *Model) Init(t *rapid.T) {
    // initialise state using *testing.T
    // ..
}

func (m *Model) Cleanup() {
}

func (m *Model) Check(t *rapid.T) {
    if 0 != 0 {
        t.Fatalf("wrong!")
    }
}

func (m *Model) Foo(t *rapid.T) {
    // Action
}

func TestPBT(t *testing.T) {
    rapid.Check(t, rapid.Run[*Model]())
}

Can I achieve what I want another way?

danwt commented 1 year ago

Essentially rapid uses reflection to call reflect.New(typ.Elem()) and init has a strict signature so there is no way to inject custom state into the state machine.

flyingmutant commented 1 year ago

Can you tell what are these libraries, and why do they need a concrete *testing.T?

danwt commented 1 year ago

Sure, e.g. cosmos/ibc-go/testing

https://github.com/cosmos/ibc-go/blob/6e6773018636c1f8650dd0f68a32ade96a7db03d/testing/chain.go#L51

Used for testing blockchains.

why do they need a concrete *testing.T

They don't but I can't necessarily do anything about that.

flyingmutant commented 1 year ago

It looks like the library you are using is making indirect calls to *testing.T.Fatalf():

https://github.com/cosmos/ibc-go/blob/6e6773018636c1f8650dd0f68a32ade96a7db03d/testing/chain.go#L245-L249

This will not work with rapid, as rapid requires that you call *rapid.T.Fatalf() instead. I think the best course of action is to submit a small PR to ibc-go which changes it to use an interface instead of a concrete type. I don't think this issue can or should be solved from the rapid's side.

danwt commented 1 year ago

Ok thanks I think you're right re this specific case ibc-go. I still think it would be nice to have a way to access an arbitrary object of the user choosing in Init.

flyingmutant commented 1 year ago

Maybe a way to forward additional arguments of Run to Init? Sounds useful, and can be done without breaking backward compatibility. Another possible design would be to simply make Run take a StateMachine object as a regular argument.