I opened see hspec/hspec#850, which captures this, but I removed hspec-golden from the example (thankfully preserving the bug).
Here is hopefully an equivalent version that shows hspec-golden’s issue:
.golden/should_fail/golden:
success content
golden-test.hs:
import Test.Hspec (around_, describe, hspec, it)
import Test.Hspec.Golden (defaultGolden)
import Test.Mockery.Directory (inTempDirectory)
main :: IO ()
main = hspec $ do
describe "golden manual" $
it "should fail" $
inTempDirectory $ do
-- do something in the temp directory
defaultGolden "should_fail" "failure content"
describe "golden around" $ around_ inTempDirectory $
it "should fail" $ do
-- do something in the temp directory
defaultGolden "should_fail" "failure content"
I expect these two test cases to fail (as “failure content” does not match “success content”).
The first case fails as expected, but the second succeeds claiming something like “first run, creating golden file”. But the golden file is created in the temp directory, and the one that exists in the repo gets ignored.
@sol posted a solution to my non-golden example in https://github.com/hspec/hspec/issues/850#issuecomment-1636503414. The gist is that the Example instances do too much in the function passed to hook (action in the hspec-golden code). So it should look something like:
instance Eq str => Example (arg -> IO (Golden str)) where
type Arg (arg -> IO (Golden str)) = arg
evaluateExample golden _ action _ = do
ref <- newIORef ""
action $ writeIORef ref <=< golden
r <- runGolden =<< readIORef ref
pure $ fromGoldenResult r
Hello @sellout Thanks for reporting this issue. I'll wrap my head around it during this week to understand the issue and the possible solution that you mentioned. Thanks!
I opened see hspec/hspec#850, which captures this, but I removed
hspec-golden
from the example (thankfully preserving the bug).Here is hopefully an equivalent version that shows hspec-golden’s issue:
.golden/should_fail/golden:
golden-test.hs:
I expect these two test cases to fail (as “failure content” does not match “success content”).
The first case fails as expected, but the second succeeds claiming something like “first run, creating golden file”. But the golden file is created in the temp directory, and the one that exists in the repo gets ignored.
@sol posted a solution to my non-golden example in https://github.com/hspec/hspec/issues/850#issuecomment-1636503414. The gist is that the
Example
instances do too much in the function passed tohook
(action
in the hspec-golden code). So it should look something like:(but I haven’t tested that)