advancedtelematic / quickcheck-state-machine

Test monadic programs using state machine based models
Other
204 stars 25 forks source link

Cleanup #346

Closed kderme closed 4 years ago

kderme commented 4 years ago

This pr adds a new cleanup function field in StateMachine. In the sequential case the cleanup is applied to the final model. In the parallel case, a possible model is created from the existing history. So, even if there is some error or an async exception, or we run each program many times using runParallelCommandsNTimes, the cleanup will be made after each execution. For this reason we have replaced MonadThrow constraints with MonadMask.

This pr adds also a new example test, called Cleanup, with many new properties and metaproperties. The test itself is pretty simple. The recourses are files (not open file handles), created and the cleanup tries to delete all files, by looking the References in the final Model. By the end of execution we test if the test folder is indeed empty an this is our property. We test different scenarios like

This pr introduces a breaking api change. Users can add the noCleanup function at the new field of StateMachine to have it work as before.

kderme commented 4 years ago

Tests probably fail because Tasty runs them in parallel, so they affect each other. I had the same issue locally, so I ran them with stack test --ta="-p Cleanup --num-threads 1" Not sure what we can do here..

stevana commented 4 years ago

Perhaps you can introduce more randomness when generating filepaths for Create? Now several tests will have file1 etc in common, but if you generate file-$32_RANDOM_ASCII_CHARS-1 they should almost never clash.

kderme commented 4 years ago

I added some randomness on the testing-folder, wo that different tests don't collide. Maybe I overdid it with the number of new tests, I seems to take again a lot of time for ci to finish, probably I should reduce them.

kderme commented 4 years ago

I think actually the ci timeouts is because lts11 doesn't find the cache and downloads everything. Actual test time seems to be the same as before.