hedgehogqa / fsharp-hedgehog

Release with confidence, state-of-the-art property testing for .NET.
https://hedgehogqa.github.io/fsharp-hedgehog/
Other
271 stars 31 forks source link

Doesn't shrink or print generated values if using for loop in property #427

Open cmeeren opened 1 year ago

cmeeren commented 1 year ago

I am running the following test in Expecto:

testCase "Blah" <| fun () ->
  Property.check <| property {
    let! xs =
      Gen.int32 (Range.exponentialBounded ())
      |> Gen.list (Range.linear 1 10)

    for i in xs do
      Expect.isLessThan i 1000 ""
  }

When this fails, it does not shrink or print the generated values:

[11:13:21 ERR] SlidingWindowRateLimiter.Blah failed in 00:00:00.0410000.
. Expected a (1556) to be less than b (1000).
   at UtilsTests.tests@61-4.Invoke(Int32 i) in C:\path\to\tests.fs:line 62
   at Hedgehog.Property.delay@26-2.Invoke(Unit x)
   at Hedgehog.Gen.delay@19-1.Invoke(Unit x)
   at Hedgehog.Random.delay@18.Invoke(Seed seed, Int32 size)
 <Expecto>ehog.Random.map@39-3.Invoke(Seed seed, Int32 size)

Replacing the loop with xs |> List.iter (fun i -> Expect.isLessThan i 1000 "") works:

[11:15:12 ERR] SlidingWindowRateLimiter.Blah errored in 00:00:00.0810000 <Expecto>
System.Exception: *** Failed! Falsifiable (after 34 tests and 6 shrinks):
[1000]
Expecto.AssertException: . Expected a (1000) to be less than b (1000).
   at Expecto.Expect.isLessThan@185.Invoke(String msg)
   at UtilsTests.tests@61-3.Invoke(FSharpList`1 xs) in C:\path\to\tests.fs:line 61
   at Hedgehog.Property.kTry@88.Invoke(a a)
This failure can be reproduced by running:
> Property.recheck "33_2529029042667730436_6746702585223722841_0101110111111101111111101111111110" <property>
   at Hedgehog.ReportModule.tryRaise(Report report)
   at Hedgehog.Property.check(Property`1 p)
   at UtilsTests.tests@56-1.Invoke(Unit unitVar0) in C:\path\to\tests.fs:line 56
   at Expecto.Impl.execTestAsync@569-1.Invoke(Unit unitVar)
   at Microsoft.FSharp.Control.AsyncPrimitives.CallThenInvoke[T,TResult](AsyncActivation`1 ctxt, TResult result1, FSharpFunc`2 part2) in D:\a\_work\1\s\src\FSharp.Core\async.fs:line 510
   at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc`2 firstAction) in D:\a\_work\1\s\src\FSharp.Core\async.fs:line 112