Fuzzing 0 ^ -1 is sometimes +Infinity and sometimes -Infinity

[nunntom]

I've recently upgraded to elm-test 2 for one of my packages which passed all tests in the old version. The new fuzzers have thrown some interesting things into the mix.

A very contrived reproduction of a real problem I ran into:

SSCCE:

infinityWeirdnessTest : Test
infinityWeirdnessTest =
    Test.describe
        "Fuzzing 0 ^ -1 is sometimes +Infinity and sometimes -Infinity"
        [ Test.fuzz (Fuzz.pair (Fuzz.floatRange -1 1) (Fuzz.floatRange -1 1))
            "This test will fail Infinity /= -Infinity"
          <|
            \( f1, f2 ) ->
                if f1 == 0 && f2 == -1 then
                    Expect.within (Expect.Absolute 0) (f1 ^ f2) (0 ^ -1)

                else
                    Expect.within (Expect.Absolute 0) (f1 ^ f2) (f1 ^ f2)
        , Test.fuzz (Fuzz.pair (Fuzz.floatRange -1 1) (Fuzz.floatRange -1 1))
            "This test will fail -Infinity /= Infinity"
          <|
            \( f1, f2 ) ->
                if f1 == 0 && f2 == -1 then
                    Expect.within (Expect.Absolute 0) (f1 ^ f2) (-1 / 0)

                else
                    Expect.within (Expect.Absolute 0) (f1 ^ f2) (f1 ^ f2)
        ]

npx elm-test --fuzz 100 --seed 150514400077418

Output:

Compiling > Starting tests

elm-test 0.19.1-revision9
-------------------------

Running 2 tests. To reproduce these results, run: elm-test --fuzz 100 --seed 150514400077418

↓ Power
↓ Fuzzing 0 ^ -1 is sometimes +Infinity and sometimes -Infinity
✗ This test will fail Infinity /= -Infinity

Given (0,-1)

    Infinity
    ╷
    │ Expect.within Absolute 0
    ╵
    -Infinity

↓ Power
↓ Fuzzing 0 ^ -1 is sometimes +Infinity and sometimes -Infinity
✗ This test will fail -Infinity /= Infinity

Given (0,-1)

    -Infinity
    ╷
    │ Expect.within Absolute 0
    ╵
    Infinity

TEST RUN FAILED

Duration: 166 ms
Passed:   0
Failed:   2

Notice both failures are given (0, -1), but in the second one the -Infinity has switched places.

Any ideas what's going on?

[avh4]

Is maybe one of them getting -0 instead of (positive) 0, but they both print as "0" so it's hidden in the error message display?

On Wed, Apr 26, 2023, 2:08 PM nunntom @.***> wrote:

I've recently upgraded to elm-test 2 for one of my packages which passed all tests in the old version. The new fuzzers have thrown some interesting things into the mix.

A very contrived reproduction of a real problem I ran into: SSCCE:

infinityWeirdnessTest : TestinfinityWeirdnessTest = Test.describe "Fuzzing 0 ^ -1 is sometimes +Infinity and sometimes -Infinity" [ Test.fuzz (Fuzz.pair (Fuzz.floatRange -1 1) (Fuzz.floatRange -1 1)) "This test will fail Infinity /= -Infinity" <| ( f1, f2 ) -> if f1 == 0 && f2 == -1 then Expect.within (Expect.Absolute 0) (f1 ^ f2) (0 ^ -1)

            else
                Expect.within (Expect.Absolute 0) (f1 ^ f2) (f1 ^ f2)
    , Test.fuzz (Fuzz.pair (Fuzz.floatRange -1 1) (Fuzz.floatRange -1 1))
        "This test will fail -Infinity /= Infinity"
      <|
        \( f1, f2 ) ->
            if f1 == 0 && f2 == -1 then
                Expect.within (Expect.Absolute 0) (f1 ^ f2) (-1 / 0)

            else
                Expect.within (Expect.Absolute 0) (f1 ^ f2) (f1 ^ f2)
    ]

npx elm-test --fuzz 100 --seed 150514400077418 Output:

Compiling > Starting tests

elm-test 0.19.1-revision9

Running 2 tests. To reproduce these results, run: elm-test --fuzz 100 --seed 150514400077418

↓ Power ↓ Fuzzing 0 ^ -1 is sometimes +Infinity and sometimes -Infinity ✗ This test will fail Infinity /= -Infinity

Given (0,-1)

Infinity
╷
│ Expect.within Absolute 0
╵
-Infinity

↓ Power ↓ Fuzzing 0 ^ -1 is sometimes +Infinity and sometimes -Infinity ✗ This test will fail -Infinity /= Infinity

Given (0,-1)

-Infinity
╷
│ Expect.within Absolute 0
╵
Infinity

TEST RUN FAILED

Duration: 166 ms Passed: 0 Failed: 2

Notice both failures are given (0, -1), but in the second one the -Infinity has switched places.

Any ideas what's going on?

— Reply to this email directly, view it on GitHub https://github.com/elm-explorations/test/issues/215, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAAJRW6OWYUFDBIRO7BH4TXDGFEXANCNFSM6AAAAAAXM7GGC4 . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[nunntom]

Interesting. Yes that does appear to be what's happening.

Seems that Fuzz.niceFloat and Fuzz.float never produce -0, but Fuzz.floatRange does.

Maybe it would be good to check for -0 and specifically print "-0" for the error?