Pass Result to NextState callback

[onepunchtech]

Sometimes the only way to know what the next state should be is if you let the system tell you. So it would be nice to be able to have the Result of Run in the NextState callback. Right now I am modifying the state in the post condition.

For example in a CRUD application you can know all of the parameters except for the unique id that gets generated by the system.

Concrete use case:

• command: create user

  • next state: save user with id to model state.
  • post condition: ensure creation response conforms

• command: get user

  • pre condition: at least one user must exist
  • post condition: ensure application response matches model state

[untoldwind]

For stateful testing I used scala-check as template. In this case: https://www.scalacheck.org/files/scalacheck_2.11-1.14.0-api/index.html#org.scalacheck.commands.Commands$Command

But that does not mean that it is set in stone.

So, if I understand correctly, you need the Result of Run as additional parameter to NextState so that your State-implementation can be immutable?

Or even more specific, the new Command-interface should look like this:

type Command interface {
    Run(systemUnderTest SystemUnderTest) Result
    NextState(state State, result Result) State
    PreCondition(state State) bool
    PostCondition(state State, result Result) *gopter.PropResult
    String() string
}

Alternatively (if just focusing on the immutability) PostCondition might return a modified state as well, like this:

type Command interface {
    Run(systemUnderTest SystemUnderTest) Result
    NextState(state State) State
    PreCondition(state State) bool
    PostCondition(state State, result Result) (State, *gopter.PropResult)
    String() string
}

Though I think the first variant is much cleaner

[onepunchtech]

I used scalacheck a couple times, and ran into the same issue. I don't remember how exactly I got around it.

Here is a similar library in haskell, which in my opinion has a better api if you are interested. http://hackage.haskell.org/package/hedgehog-0.6/docs/Hedgehog-Internal-State.html#t:Callback. You can see there that the state, input, and result is passed to the update callback.

Back to the original point, yes, I think that the first one is cleaner, and regardless of keeping state immutable (which isn't a super important thing for me to do in go) I think that separating the concerns where my nextState callback is concerned with updating state, and my postCondition callback is only worried about making assertions.

[untoldwind]

I experimented a bit and just realized, that NextState is pretty essential for the shrinker. More precisely: PreCondition and NextState are used to generate valid Command sequences without actually altering/invoking the SystemUnderTest. I suppose that's one of the reasons why scala-check is doing it this was.

Adding the result to NextState would mean that during the creation of the command sequences the SystemUnderTest would have to be resetted and applied on every try before the actual test even runs. That's not only a major redesign but potentially also a huge performance drawback.

Potentially it would make more sense to use the haskell API as template for an extra command package.

[untoldwind]

The main trick of the haskell variant seems to be that they distinguish between a symbolic state and a concrete state, which is pretty nice when used with polymorphic functions.

A go-ish version might look like this (I'm dropping the command input's for now, that would just add even more complexity):

package commands2

// SystemUnderTest resembles the system under test, which may be any kind
// of stateful unit of code
type SystemUnderTest interface{}

// Symbol resembles a symbol in the state
type Symbol interface{}

// SymbolicState resembles the symbolic state (i.e. a state without concrete values)
type SymbolicState map[Symbol]bool

// ConcreteState resembles the concrete state (i.e. a state with concrete values)
type ConcreteState map[Symbol]interface{}

// Output resembles the Output of a command that may or may not be checked
type Output interface{}

// Command is any kind of command that may be applied to the system under test
type Command interface {
    // Execute the command
    Execute(systemUnderTest SystemUnderTest) Output

    Require(state SymbolicState) bool

    UpdateSymbolic(state SymbolicState) SymbolicState

    UpdateConcrete(state ConcreteState, output Output) ConcreteState

    Ensure(prev ConcreteState, next ConcreteState) bool
}

I.e. I declared that a State is always a collection/map of symbols with potential values. Since go not even remotely has such a thing like polymorphic functions (not that I'm aware of at least), the implementor has to provide an Update for the symbolic state and the concrete state.

In your original example it is pretty clear that a GetUser command always require a previous CreateUser command. So you have a symbol "userId" as part of the state. UpdateSymbolic for CreateUser would just add this symbol, while UpdateConcrete would add a concrete userId to it.

I think that something like this might solve your problem.