flyingmutant / rapid

Rapid is a modern Go property-based testing library
https://pkg.go.dev/pgregory.net/rapid
Mozilla Public License 2.0
579 stars 25 forks source link

Allow to give a specializer when running the state machine #40

Closed ibizaman closed 1 year ago

ibizaman commented 1 year ago

Allow to give a specializer when running the state machine

The use case is the following: we want to be able to test multiple implementations with the same state machine. Adding this specialize function allows to tweak the state machine after creation.

A concrete example for us is testing database accessors. We have one struct with methods that accesses the real database and we're currently using rapid to test it with a state machine, and it's wonderful. But then, we want to create a stub that instead uses an in-memory data store and will be used in unit tests. Right now we're using mocks but would like to transition to stubs as maintaining the mocks gets hard now. But of course, we want to make sure the new stub matches the methods accessing the real database. This PR allows us to do the following:

We can have multiple tests using the same state machine living at the same time:

func TestUser_DB(t *testing.T) {
    t.Parallel()

    rapid.Check(t, rapid.RunWithSpecializer[*stateMachine](func(t *rapid.T, m interface{}) {
        sm := m.(*stateMachine)

        sm.user, sm.close = setupUserDatabase(t)
    }))
}

// Testing the stub
func TestUser_Stub(t *testing.T) {
    t.Parallel()

    rapid.Check(t, rapid.RunWithSpecializer[*stateMachine](func(t *rapid.T, m interface{}) {
        sm := m.(*stateMachine)

        sm.user = setupStub(t)
    }))
}

// Testing a migration to another driver
func TestUser_DB_other_driver(t *testing.T) {
    t.Parallel()

    rapid.Check(t, rapid.RunWithSpecializer[*stateMachine](func(t *rapid.T, m interface{}) {
        sm := m.(*stateMachine)

        sm.user, sm.close = setupUserDatabaseOtherDriver(t)
    }))
}

// Testing a migration to another version of a driver
func TestUser_DB_v2(t *testing.T) {
    t.Parallel()

    rapid.Check(t, rapid.RunWithSpecializer[*stateMachine](func(t *rapid.T, m interface{}) {
        sm := m.(*stateMachine)

        sm.user, sm.close  = setupUserDatabaseV2(t)
    }))
}

And then have a common state machine for everything:

type userMethods interface {
    Find()
    Insert()
}

type stateMachine struct {
    user userMethods
}

func (sm *stateMachine) Find(t *rapid.T) {
    sm.user.Find()
}

func (sm *stateMachine) Insert(t *rapid.T) {
    sm.user.Insert
}

What do you think of this?

flyingmutant commented 1 year ago

Hm, maybe this can be achieved by wrapping the "base" state machine type in several "derived" types which override the Init() with the required initialization?

type stateMachine struct {
    user userMethods
}

type dbStatMachine struct {
    stateMachine
}

func (m *dbStateMachine) Init(t *rapid.T) {
    sm.user, sm.close = setupUserDatabase(t)
}

// ...
ibizaman commented 1 year ago

Take this with a grain of salt because we didn't yet upgrade this package to use generics. But this works! The check must be called like this:

rapid.Check(t, rapid.Run(& dbStateMachine{stateMachine{}}))

I'll report back when we upgraded to see if it still holds.