search

advancedtelematic / quickcheck-state-machine

Test monadic programs using state machine based models
Other
203 stars 25 forks source link

Property Result in case of exceptions #326

Open kderme opened 5 years ago

kderme commented 5 years ago

At the moment runParallelCommandsNTimes and prettyParallelCommands return a passing Result even when there is some thrown exception on semantics. Maybe we should discuss which should be the default behaviour.

Also the equivalent NParallel.. commands may encounter executeCommands: impossible (for example when an exception during a MemoryReferene Create fails) because this https://github.com/advancedtelematic/quickcheck-state-machine/pull/317 is not adopted yet.

stevana commented 5 years ago

What do you mean by "return a passing Result"?

stevana commented 5 years ago

Also the equivalent NParallel.. commands may encounter executeCommands: impossible (for example when an exception during a MemoryReferene Create fails) because this #317 is not adopted yet.

Yes, we should implement the analogue of #317 for the NParallel... functions.

kderme commented 5 years ago

What do you mean by "return a passing Result"?

The property doesn't fail

stevana commented 5 years ago

The parallel property shouldn't necessarily fail if an exception is thrown on one of the threads. It should only fail if we cannot complete the history (#317) in any way that linearises.

Arguably we should either make runParallelCommands' (which completes histories) the default way to run the parallel property and remove runParallelCommands, or keep both runParallelCommands and runParallelCommands' and have runParallelCommands fail if there's any exception (since that function doesn't complete histories).

Is that what you meant or did you mean something else?

stevana commented 5 years ago

Another way to think of it is that runParallelCommands should be used when there's no fault injection (exceptions), while runParallelCommands' should be used when there's fault injection.

kderme commented 5 years ago

Yes, this is what I meant. I personally agree a bit more with the second option: make runParallelCommands fail on exceptions, since it doesn't complete histories.

kderme commented 5 years ago

Another way to think of it is that runParallelCommands should be used when there's no fault injection (exceptions), while runParallelCommands' should be used when there's fault injection.

Yes exactly.

stevana commented 5 years ago

I personally agree a bit more with the second option...

Yeah, I think you're right -- that's probably the better option.