NorfairKing / sydtest

A modern testing framework for Haskell with good defaults and advanced testing features.
113 stars 25 forks source link

Complete stack trace in test localisation #78

Open guibou opened 2 months ago

guibou commented 2 months ago

Hello,

When sydtest is doing its reporting, it will report the file localisation of the first it found (if HasCallStack of sub function are correctly set).

However, if the it is itself inside an utility function, the localisation won't be useful.

See for yourself:

{-# LANGUAGE PartialTypeSignatures #-}
{-# OPTIONS_GHC -Wno-partial-type-signatures #-}

import GHC.Stack
import Test.Syd

foo :: HasCallStack => Int -> _
foo i = it "test" $ i `shouldBe` 2

main = sydTest $ do
  describe "yo" $ do
    foo 1
  describe "to" $ do
    foo 2
$ runhaskell Main.hs 
WARNING: Only one CPU core detected, make sure to compile your test suite with these ghc options:
         -threaded -rtsopts -with-rtsopts=-N
         (This is important for correctness as well as speed, as a parallel test suite can find thread safety problems.)
Tests:

yo
  ✗ test                                                                   0.01 ms
    Retries: 3 (does not look flaky)
to
  ✓ test                                                                   0.00 ms

Failures:

    Main.hs:8
  ✗ 1 yo.test
      Retries: 3 (does not look flaky)
      Expected these values to be equal:
      Actual:   1
      Expected: 2

  Examples:                     4
  Passed:                       1
  Failed:                       1
  Sum of test runtimes:         0.00 seconds
  Test suite took:              0.00 seconds

The localisation is Main.hs:8, which is the it localisation.

It would be great if sydtest was able to also show the rest of the trace, in order to quickly localize the offending test.

NorfairKing commented 2 months ago

@guibou PR welcome. In the meantime you can use withFrozenCallStack inside your utility function.

guibou commented 2 months ago

Thank you for the workaround.

I'll have a look for the PR. Thank you.