hspec / hspec-hedgehog

A library to integrate hedgehog tests into your hspec test suite.
https://hackage.haskell.org/package/hspec-hedgehog
Other
28 stars 6 forks source link

Labeling doesn't seem to work #12

Closed MasseR closed 1 year ago

MasseR commented 4 years ago

I've been trying out hedgehog and noticed that labeling doesn't seem to work. Take a look at this example:

{-# LANGUAGE TemplateHaskell #-}
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE RecordWildCards #-}
{-# LANGUAGE OverloadedStrings #-}
module Main where

import Control.Monad (when)

import qualified Hedgehog.Gen as Gen
import qualified Hedgehog.Range as Range

import Test.Hspec
import Test.Hspec.Hedgehog

prop_reverse :: Property
prop_reverse = property propRev

propRev :: Monad m => PropertyT m ()
propRev = do
  xs <- forAll $ Gen.list (Range.linear 0 100) Gen.alpha
  when (length xs > 50) $ label "Big list"
  reverse (reverse xs) === xs

baseHedgehog :: IO ()
baseHedgehog = do
  _ <- checkParallel $$(discover)
  pure ()

hspecHedgehog :: IO ()
hspecHedgehog = hspec $
  describe "foo" $
    it "works" $ hedgehog propRev

main :: IO ()
main = do
  baseHedgehog
  hspecHedgehog

It's defining a simple property with a dummy label. When using the base hedgehog runner, it labels properly, but when using the hspec runner there are no labels:

━━━ Main ━━━
  ✓ prop_reverse passed 100 tests.
    Big list 15% ███·················
  ✓ 1 succeeded.

foo
  works

Finished in 0.0069 seconds
1 example, 0 failures
EarthCitizen commented 4 years ago

I was getting ready to open an issue for exactly this. I will provide an additional example:

prop_with_coverage :: Property
prop_with_coverage =
 property $ do
   match <- forAll Gen.bool
   cover 30 "True" $ match
   cover 30 "False" $ not match

tests :: IO Bool
tests =
  checkParallel $ Group "Test.Example" [
      ("prop_with_coverage", prop_with_coverage)
    ]

Outputs:

━━━ Test.Example ━━━
  ✓ prop_with_coverage passed 100 tests.
    True  49% █████████▊·········· ✓ 30%
    False 51% ██████████▏········· ✓ 30%
  ✓ 1 succeeded.

So, it is possible to somehow capture this output and display it when hspec runs?

simfleischman commented 3 years ago

It seems one could just print this value (even using something as simple as putStrLn)

https://github.com/parsonsmatt/hspec-hedgehog/blob/eb617d854542510f0129acdea4bf52e50b13042e/src/Test/Hspec/Hedgehog.hs#L200

which would output the coverage cases.

Since the failure case already outputs this: https://github.com/parsonsmatt/hspec-hedgehog/blob/eb617d854542510f0129acdea4bf52e50b13042e/src/Test/Hspec/Hedgehog.hs#L211

maybe it would be better to only do it on OK.

And then I wonder would it be a good idea to always output this information? or have some way of including it or omitting it?

Would that mean adding a new function like hedgehog but that takes an extra argument whether the Hedgehog test output is desired in the success case? Or env var? Something else?

sol commented 1 year ago

https://github.com/parsonsmatt/hspec-hedgehog/pull/30 fixes this.