Abstract state machine testing

[moodmosaic]

The F# version should also support State Machine Testing similar to the Haskell version.

See https://github.com/hedgehogqa/haskell-hedgehog/pull/89, and at least up to https://github.com/hedgehogqa/haskell-hedgehog/pull/96.

To have a solid foundation to build on, we first need to have Tree and Gen data types and combinators in line with the Haskell version, as per https://github.com/hedgehogqa/fsharp-hedgehog/issues/111 and https://github.com/hedgehogqa/fsharp-hedgehog/issues/89.

[moodmosaic]

[..] need to have Tree and Gen data types and combinators in line with the Haskell version.

For Gen, look at the discussion in #89.

[moodmosaic]

Fun experiment: https://github.com/moodmosaic/hedgehog-inline-csharp-testing

Example of using haskell-hedgehog's model-based state machine testing together with clr-inline (C#).

[moodmosaic]

We should just do this the 'modern' way by having Hedgehog construct a random structure by executing generated commands, as shown in https://github.com/moodmosaic/hedgehog-stateful-app-testing/blob/master/test/test.hs

A simplified version of the above in F# would be to have Hedgehog generate a list of commands which you then apply into both the system under test and the state representing the model, and then you compare the two.

open System.Collections.Generic

open Hedgehog
open Swensen.Unquote

[<Struct>]
type UserId = 
    | UserId of string

type User = 
    { id : UserId
      name : string
      age : int }

type IUserOperations = 
    abstract AddUser : User   -> unit
    abstract GetUser : UserId -> Option<User>
    abstract DelUser : UserId -> unit

type UserSystem () = 
    let users =
        Dictionary<UserId, User> ()

    member _.UserCount = users.Count

    interface IUserOperations with
        member _.AddUser u =
            users.[u.id] <- u
        member _.GetUser id = 
            match users.TryGetValue id with
            | (true, user) -> Some user
            |            _ -> None
        member _.DelUser (UserId id) = 
            if id.Contains "*" // Uh-oh: catastrophic bug!
            then
                users.Clear ()
            else
                users.Remove (UserId id) |> ignore

type Operation = 
    | Add of User
    | Get of UserId
    | Del of UserId

let applyOp (op : Operation) (handler : IUserOperations) = 
    match op with
    | Add user -> handler.AddUser user
    | Get name -> handler.GetUser name |> ignore
    | Del name -> handler.DelUser name

type UserCountModel () = 
    let expected =
        HashSet<UserId> ()
    member _.Verify (actual : UserSystem) =
        expected.Count =! actual.UserCount
    interface IUserOperations with
        member _.AddUser user = expected.Add user.id |> ignore
        member _.DelUser name = expected.Remove name |> ignore
        member _.GetUser name = None

[<EntryPoint>]
let main argv =
    Property.print <| property {
        let state = UserCountModel ()
        let model = UserSystem ()
        let! operations = GenX.auto<Operation list>
        operations |> List.iter (fun op -> applyOp op model; applyOp op state)
        state.Verify model
    }
    0 // Return an integer exit code.

[ghost]

@moodmosaic what would we need to implement from where we are now to get this done?

[moodmosaic]

Now, in 2021, I would try to avoid adding a dedicated API for state-machine testing.

Instead, I would add the example code above in the README/docs, or in its own test-suite, and be done with it.

As @jacobstanley also describes in his post:

"The state-machine testing libraries aimed at solving this problem use complicated types that are overwhelming and hard to use, especially for newbies."

This has been true also IME, in various shops/groups I've helped with PBT in the past.

---

Perhaps, an even better example/scenario would be to port Jake's post in F# (or C#) Hedgehog and add also that in the README/docs, or in a test-suite, and port any bits missing from Haskell in order to achieve that.