well-typed / falsify

Other
36 stars 5 forks source link

WIP: First draft of testPropertyIO. Needs refactor before merge #62

Open voidus opened 1 year ago

voidus commented 1 year ago

Very ugly proof of concept to address #57

This is a huge mess of copy/paste/mess-with, but it seems to work and I wanted to share it.

I definitely want to look into making assert (or maybe a new similar function) work in IO so we can get the same rich predicates that we're used to.

I think wrapping the predicate result in a custom exception and handling that specifically is probably possible. We could also add a type class so we can use assert both in IO and Property, but I'm not sure if that's actually a good idea yet.

The type of testPropertyIO is TestName -> Property (IO ()) -> TestTree, which forces the property into a two-phase model. Once we've entered IO-land, we can't use generators anymore.

test:

    testPropertyIO "foo" $ do
        x <- gen $ Gen.int $ Range.between (0, 500)
        return $
            when (22 < x && x < 40) $ throwIO (userError "aaaAAAaaAAAa")

output:

> cabal run demo:exe:demo -- --falsify-verbose
demo
  foo: FAIL
    failed after 1 successful test and 2 shrinks
    aaaAAAaaAAAa
    CallStack (from HasCallStack):
      error, called at demo/Main.hs:23:39 in main:Main

    Logs for complete shrink history:

    Step 1
    generated 29 at CallStack (from HasCallStack):
      gen, called at demo/Main.hs:21:14 in main:Main

    Step 2
    generated 26 at CallStack (from HasCallStack):
      gen, called at demo/Main.hs:21:14 in main:Main

    Step 3
    generated 23 at CallStack (from HasCallStack):
      gen, called at demo/Main.hs:21:14 in main:Main

    Use --falsify-replay=01c366d9f1f14fe8deb259272f6b6dce95 to replay.

1 out of 1 tests failed (0.00s)
edsko commented 1 year ago

I have limited time at the moment (well, actually, always..) but if you feel this this is ready for me to take a look, let me know :)

edsko commented 10 months ago

Marked this as WIP. Feel free to ping me when you think this is ready for review :)