sorellabs / claire

[Unmaintained: please use jsverify instead] A property-based testing library for clearly specifying code invariants and behaviour.
MIT License
79 stars 4 forks source link

Move generator to a separate library #24

Open robotlolita opened 11 years ago

robotlolita commented 11 years ago

Let's just implement generators as generic monadic streams in a separate library, which provides core combinators to work with them, like the cool and awesome interpose, iterate, etc, etc, etc.

This keeps the implementation of Claire simple, helps other people, and we can use some kind of ad-hoc polymorphism to get shrinking done faster, better, stronger ;3

kennknowles commented 11 years ago

+1

I have also had the thought that a single library of combinators could operate on QuickCheck's Gen and SmallCheck's Series at the same time. For example Array(Int) could be a random array of ints or the exhaustive stream of all arrays of ints. Basically just interpreting type-like syntax, but with the power to write more arbitrary generators. (This is sort of my end dream if I ever get time to polish python-doublecheck and combine with python-rightarrow)

robotlolita commented 11 years ago

Working on it at http://github.com/killdream/lazysex. I guess SmallCheck-style could be supported through some kind of checking method on the Property object (I've missed it from Claire for a huge file I needed to process this week):

// QuickCheck generator
var arbInt = iterate(function() { return (Math.random() * Math.pow(2, 32)) | 0 }, 0)
forAll(arbInt).satisfy(...).quickTest() // or some better name

// SmallCheck series
var int = iterate(function(n){ return n + 1 <= MAX_INT?  n + 1 : Nothing }, 0)
forAll(int).satisfy(...).exhaustiveTest() // or some better name

I'm not sure how the same generators would fit both types of test efficiently.

kennknowles commented 11 years ago

What I mean is one level of abstraction up from that. For atomic values, you still have to write both, but for combinators (other than frequency, etc) you don't. Something like this.

data Nat = Zero | Succ Nat

anyNat :: Generate g => g Nat
anyNat = choice [unit Zero, Succ `appliedTo` anyNat] -- In Haskell this is obvs derivable

-- And this works because of something like this...

class Generate g where
    choice :: [g a] -> g a
    list :: g a -> [g a]
    dict :: Map a (g b) ] -> g (Map a b)
    unit :: a -> g a
    appliedTo :: (a -> b) -> g a -> g b
    ... -- other combinators

instance Generate Gen where    ...

instance Generate Series where    ...

forAll :: Generate g => (a -> TestResult) -> g a -> TestResult
robotlolita commented 11 years ago

Hm, so whether we perform an exhaustive test or a quick check would be defined by the type of the generator?

kennknowles commented 11 years ago

Sorry, I just said something completely incorrect so you probably got an email notification. But it could be made correct.

The best would be for the overloading to be on the return value so that the test suite decides. Such as instead of TestResult have it return g TestResult. Without types, this would probably be part of the test config passed in.