search

advancedtelematic / quickcheck-state-machine

Test monadic programs using state machine based models
Other
203 stars 25 forks source link

Gracefully stop when no precondition holds #254

Closed mrBliss closed 5 years ago

mrBliss commented 5 years ago

Imagine that an (expected) error is thrown during the tests and the tested system is now in a state where it can no longer process new commands. In that case we should be able to gracefully stop the test in progress.

Simply stop the test when no precondition holds instead of reporting a deadlock error. Labelling can be used to detect such deadlocks.

This is an alternative to #253.

mrBliss commented 5 years ago

The non-determinism is not an issue (anymore) for me, I just want to be able to stop a test.

So:

model Symbolic -> Maybe (Gen (cmd Symbolic))

is a perfectly good alternative for me.

Do you want me to make a PR for this or do you want to wait for #242?

stevana commented 5 years ago

How soon do you need this?

Merging #242 might take some time. I don't exactly know how to provide the user with the choice of using the old size based generation or the Markov chain based one. Perhaps by splitting the StateMachine record into two, one simple interface (size based and no mock* nor perhaps invariant) and an advanced/experimental interface (Markov chain based and with mock and invariant). The idea would be that the simple interface is enough for the examples in test/ and that it's relatively stable, while the advanced/experimental is more of a playground for new ideas where the interface might change more often. But I don't see how to do this without a lot of code duplication though (but perhaps that's not such a big deal).

If you can wait or use the #242 branch, then I'd prefer to not change the type of generator. If not, then I'd also be happy to merge the Maybe (Gen (cmd Symbolic) change, but maybe I'd revert it once #242 gets merged to keep the simple interface simple.

Thoughts?

*: mock is confusing, https://github.com/advancedtelematic/quickcheck-state-machine/issues/236, and not necessary as can be witnessed by old versions of this library (unless you need responses to be able to return multiple references).

mrBliss commented 5 years ago

I think we need this before the Markov-chain stuff is finished.

I'll send a PR for the change (that you may revert later once #242 is merged).

It would be great if you made a release that includes both the PR that I'll open and Edsko's #255.

edsko commented 5 years ago

This is neither here nor there, but..

*: mock is confusing, #236

Actually, it's not that confusing, as I hope to show in my blog post :)

stevana commented 5 years ago

It would be great if you made a release that includes both the PR that I'll open and Edsko's #255.

No problem.

Actually, it's not that confusing, as I hope to show in my blog post :)

Looking forward to reading it!