search

hedgehogqa / haskell-hedgehog

Release with confidence, state-of-the-art property testing for Haskell.
677 stars 107 forks source link

Tests without generators #121

Open chris-martin opened 7 years ago

chris-martin commented 7 years ago

So, Hedgehog is a property testing library, yes, but - It's also just a really great general-purpose test runner, and I've been using it for things that aren't actually property tests. The downside is that my tests run a hundred times, when they only need to run once.

Is there a quick way to modify a property so that it only checks once?

And, I don't know if this is reasonable to ask, but: Could Hedgehog be clever enough to detect whether a property ever uses a generator, and only recheck if it does?

tmcgilchrist commented 7 years ago

I’d typically use the “once” function to just run a property that doesn’t use generators. Though it doesn’t seem the function is in hedgehog like it was in Jack.

Not sure about adding detection of tests that don’t use generator.

On Fri, 6 Oct 2017 at 6:17 am, Chris Martin notifications@github.com wrote:

So, Hedgehog is a property testing library, yes, but - It's also just a really great general-purpose test runner, and I've been using it for things that aren't actually property tests. The downside is that my tests run a hundred times, when they only need to run once.

Is there a quick way to modify a property so that it only checks once?

And, I don't know if this is reasonable to ask, but: Could Hedgehog be clever enough to detect whether a property ever uses a generator, and only recheck if it does?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/hedgehogqa/haskell-hedgehog/issues/121, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKbuQKbD8eF5gSF3tj-RIy7Y9P7Y8TCks5spStfgaJpZM4PvkSl .

nhibberd commented 7 years ago

The combinator is withTests 1, e.g. https://github.com/hedgehogqa/haskell-hedgehog/blob/d12b3e8d2150cd51e4ea89b1bb1727a3beceed2f/hedgehog-example/test/Test/Example/Basic.hs#L42

tmcgilchrist commented 7 years ago

Cheers, I thought it'd be there somewhere.

On Fri, Oct 6, 2017 at 7:59 AM, Nick Hibberd notifications@github.com wrote:

The combinator is withTests 1, e.g. https://github.com/hedgehogqa/ haskell-hedgehog/blob/d12b3e8d2150cd51e4ea89b1bb1727 a3beceed2f/hedgehog-example/test/Test/Example/Basic.hs#L42

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/hedgehogqa/haskell-hedgehog/issues/121#issuecomment-334590407, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKbuTztZOUjfA5Hu3KlyiBafcfWs_Mnks5spUMsgaJpZM4PvkSl .

kindaro commented 7 years ago

Why not make once an alias to withTests 1 then?

jacobstanley commented 7 years ago

I guess I figured having once was redundant, I tried not to have multiple ways of doing the same thing is all.

jacobstanley commented 7 years ago

And, I don't know if this is reasonable to ask, but: Could Hedgehog be clever enough to detect whether a property ever uses a generator, and only recheck if it does?

I can imagine this could possibly be added in to the Test monad but I'm not sure if it's worth the complexity.

chris-martin commented 7 years ago

I don't mind not having a once function in hedgehog, but if it doesn't I think the withTests documentation should discuss this use case to make the solution more discoverable via search.

jacobstanley commented 7 years ago

Yeah I completely agree

jacobstanley commented 7 years ago

Fwiw, I would add once if enough people wanted it, but I've been trying to keep the API as thin as possible initially so that we don't grow a bunch of unnecessary functions that end up not being widely used.

tmcgilchrist commented 7 years ago

Having once seems pretty special case to me. Having better discoverability on the right way to do this in hedgehog would be better IMHO

On Fri, 6 Oct 2017 at 9:21 am, Jacob Stanley notifications@github.com wrote:

Fwiw, I would add once if enough people wanted it, but I've been trying to keep the API as thin as possible initially so that we don't grow a bunch of unnecessary functions that end up not being widely used.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/hedgehogqa/haskell-hedgehog/issues/121#issuecomment-334608402, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKbufLM5SfOPTNXv3ADgUs2tS8RBF-Zks5spVZwgaJpZM4PvkSl .

fieldstrength commented 4 years ago

Just found this issue while wondering the same.

To me, something like the once function feels quite helpful. Writing withTests 1 is easy enough, but it does look and feel wrong. It seems like the API is telling me that I'm doing something hacky or unsupported. I've never noticed the comment added to withTests that would have told me this is actually the intended way to achieve this functionality.

I'm a big proponent of property-based testing, but in practice I continue to write a lot of example-based tests, so from my perspective its quite valuable for this not to be too awkward to achieve.

(Background: I use Hedgehog a lot at work (we're quite happy with it!) where we have some utility definitions that accomplish this. Now I'm adding it to a personal project and noticing this as a friction/confusion point.)

moodmosaic commented 4 years ago

I use Hedgehog a lot at work (we're quite happy with it!)

:+1: :heart_eyes:

where we have some utility definitions that accomplish this

:thinking: say we add once, then another request comes in for adding twice (or similar). Where do we draw the line?

In 88% of the cases we do come up with some utility functions, and I think that's where once belongs. It's like some sort of a rather special case for a property-based testing library.

In general though, just exposing a single way of doing things makes the API easier to learn for newcomers, and (often) explicit is better than implicit.