Some way to check that all state machine condition actions were called

[pierre-fastly]

It would be great if we could assert that, at least, all conditional action got ran without error (which we do already) and without them calling Skip().

The reasoning is that some of those conditional action need a precondition to be fulfilled before being able to run. We would like to assert that the precondition got fulfilled at least once by asserting that the conditional action succeeded without calling Skip.

For a concrete example, let's think about database methods testing. Let's pick a Find function, for which we can either expect the function to return an item or an error if the item does not exist. Ideally, we'd like to test at least each case once. The best we could come up with currently is to have two conditional actions - FindExisting(*rapid.T) and FindNotexisting(*rapid.T) - which does what they advertise. Now, the FindExisting conditional action has a pre-condition which is there must at least be one item in the database. If not, it calls Skip. And so in this example, we would want to be sure that the FindExisting conditional action could succeed.

Of course, this implies the tests could get flaky, so this behavior should be opt-in. And we could be fancy too by passing in the names of the conditional actions you want to assert were called successfully. But even just asserting that all conditional actions were called would be nice already.

This is a contrived example: https://go.dev/play/p/PO2g_XAOjXF copied here for convenience:

type stateMachine struct {
    precondition int
}

func (sm *stateMachine) WaitPrecond(t *rapid.T) {
    if sm.precondition < 100 {
        t.Skip()
    }
    panic("uh oh")
}

func (sm *stateMachine) SetPrecond(t *rapid.T) {
    sm.precondition++
}

func (sm *stateMachine) Check(t *rapid.T) {
}

func TestStateMachine(t *testing.T) {
    rapid.Check(t, rapid.Run[*stateMachine]())
}

[flyingmutant]

In general, satisfying arbitrary preconditions is of course impossible :-) However, we can track inside the state machine type if the condition was satisfied, and use that (how exactly?) inside Cleanup() or similar function that gets called at the very end (which may need a *rapid.T argument, maybe? as in #34).

The question is, how would you like to react? Fail the test? Or consider this test run as invalid?

[ibizaman]

I think with #34 we could build this up ourselves. We could maybe even build #30 ourselves. Not saying this package shouldn't provide something out of the box, but at least we could get start fiddling with it and think about what makes sense. Thanks for the tip here!

Would you be okay for a PR for #34 ?

[flyingmutant]

Would you be okay for a PR for #34 ?

Yes, although it is not clear yet what the best name would be. Cleanup is probably the easiest choice.