Template Haskell Splice from readme $$(discover) not working in literate Haskell

[Maik-Lesch]

I have tried to use hedgehog for a tiny bit of property-based testing in stack ghci as described in your README.

If I define the tests :: IO Bool manually, the test is correctly run and fails if I deliberately make it fail by, say, adding 1 on one side of the equation of prop_crossSum.

test-manual.lhs

> -- just administrative boilerplate for hedgehog testing
> {-# LANGUAGE TemplateHaskell #-}
> {-# LANGUAGE OverloadedStrings #-}
> module Assignment04 where
> import           Hedgehog
> import qualified Hedgehog.Gen as Gen
> import qualified Hedgehog.Range as Range

> crossSum :: Integer -> Integer -> Integer
> prop_crossSum :: Property
> prop_crossSum = property $ do
>   number <- forAll $ Gen.integral $ (Range.linear 1 100000)
>   base <- forAll $ Gen.integral $ (Range.linear 2 100)
>   crossSum base number === crossSum base (base*base*number)
> crossSum base num = foldr (+) 0 (go base (abs num) [])
>   where
>     go b n digits =
>       if (n == 0)
>       then digits
>       else go b ((n - (n `mod` b)) `div` b) ((n `mod` b):digits)

> tests :: IO Bool
> tests = checkParallel $ Group "Test.Assignment04" [
>   ("prop_crossSum", prop_crossSum)]

However, I find the explicit enumeration of tests somewhat inelegant, since I might forget to register a test for a different piece of code.

Your README provides a handy solution using template Haskell, to match all functions named with the prefix prop_ with $$(discover).

However, trying this creates a function tests, which still passes but no longer fails when I deliberately make prop_crossSum fail.

test-template.lhs

> -- just administrative boilerplate for hedgehog testing
> {-# LANGUAGE TemplateHaskell #-}
> {-# LANGUAGE OverloadedStrings #-}
> module Assignment04 where
> import           Hedgehog
> import qualified Hedgehog.Gen as Gen
> import qualified Hedgehog.Range as Range

> crossSum :: Integer -> Integer -> Integer
> prop_crossSum :: Property
> prop_crossSum = property $ do
>   number <- forAll $ Gen.integral $ (Range.linear 1 100000)
>   base <- forAll $ Gen.integral $ (Range.linear 2 100)
>   crossSum base number === crossSum base (base*base*number)
> crossSum base num = foldr (+) 0 (go base (abs num) [])
>   where
>     go b n digits =
>       if (n == 0)
>       then digits
>       else go b ((n - (n `mod` b)) `div` b) ((n `mod` b):digits)

> tests :: IO Bool
> tests = checkParallel $$(discover)

Is this an issue of your README or of my understanding? How would I make template haskell recognize prop_crossSum or any other prop_* correctly?

(Sorry if this does not conform to your convention - this is my first issue)

[moodmosaic]

Thank you for reporting this. The issue is that ghci runs in interpreted mode, which doesn't support TH splices like $$discover. This means your properties aren't being discovered, so tests always pass (even when they should fail).

To fix this, perhaps you need to enable object code compilation in ghci so that TH can execute properly:

stack ghci --ghc-options -fobject-code

Or, if you're already in ghci, set the flag and reload:

:set -fobject-code
:reload

This tells ghci to compile modules to object code, allowing. After that, your prop_* functions should be correctly discovered.

[Maik-Lesch]

Thank you very much for your response. The TH splice still does not work with the options you gave on my setup but I just discovered some outdated libraries in my system. I will clean up my setup and report back once I definitively know whether the options you gave work.

[Maik-Lesch]

I am now confident, that the options you provided are loaded on my setup (ghci creates .o files) but the TH splice continues not discovering the tests.

[moodmosaic]

This scenario starts to look like this GHC stage restriction post.

When you use a TH splice like $$discover, it runs in an earlier compilation stage. As a result, it can't see definitions in the same module because they haven't been compiled yet.

The discover function scans the module for properties named prop_*. Due to the staging restrictions, discover can't see these properties if they're defined in the same module where discover is called.

---

Could you try splitting your code into two modules?

1. Properties Module

   Create a file Assignment04/Properties.lhs:

   {-# LANGUAGE TemplateHaskell #-}
   module Assignment04.Properties where
   
   import Hedgehog
   import qualified Hedgehog.Gen as Gen
   import qualified Hedgehog.Range as Range
   
   crossSum :: Integer -> Integer -> Integer
   crossSum base num = foldr (+) 0 (go base (abs num) [])
    where
      go b n digits
        | n == 0    = digits
        | otherwise = go b (n `div` b) ((n `mod` b) : digits)
   
   prop_crossSum :: Property
   prop_crossSum = property $ do
    number <- forAll $ Gen.integral (Range.linear 1 100000)
    base   <- forAll $ Gen.integral (Range.linear 2 100)
    crossSum base number === crossSum base (base * base * number)

2. Tests Module

   Create a file Assignment04/Tests.lhs:

   {-# LANGUAGE TemplateHaskell #-}
   module Assignment04.Tests where
   
   import Hedgehog
   import qualified Assignment04.Properties
   
   tests :: IO Bool
   tests = checkParallel $$(discover)

(Make sure Assignment04.Tests imports Assignment04.Properties where your properties are defined.)