search

racket / rackunit

Other
18 stars 34 forks source link

Bad error message in check-within #165

Closed matteo-daddio closed 10 months ago

matteo-daddio commented 1 year ago

Hello, I think the error message in check-within is not correct.

The error message of check-= is clear:

> (check-= 1.0 1.01 0.005 "I fail")
--------------------
FAILURE
name:       check-=
location:   eval:12:0
params:     '(1.0 1.01 0.005)
message:    "I fail"
--------------------

but the error message of check-within is:

> (check-within (list 6e+23 10.0) (list 6.02e+23 9.8) 0.05)
--------------------
FAILURE
name:       check-within
location:   eval:16:0
actual:     '(6e+23 10.0)
expected:   '(6.02e+23 9.8)
--------------------

> (check-within (hash 'C 18 'F 64) (hash 'C 25 'F 77) 10)
--------------------
FAILURE
name:       check-within
location:   eval:17:0
actual:     '#hash((C . 18) (F . 64))
expected:   '#hash((C . 25) (F . 77))
--------------------

The second argument isn't the expected result. Is this wrong?

sorawee commented 12 months ago

Would "expected (within the tolerance)" be better?

AlexKnauth commented 12 months ago

Would just adding a tolerance: field below expected: be enough?

matteo-daddio commented 12 months ago

I think there aren't expected results, but only parameters. The error message should be:

> (check-within (list 6e+23 10.0) (list 6.02e+23 9.8) 0.05)
--------------------
FAILURE
name:       check-within
location:   eval:16:0
params:     '((6e+23 10.0)
              (6.02e+23 9.8)
              0.05)
--------------------

> (check-within (hash 'C 18 'F 64) (hash 'C 25 'F 77) 10)
--------------------
FAILURE
name:       check-within
location:   eval:17:0
params:     '(#hash((C . 18) (F . 64))
              #hash((C . 25) (F . 77))
              10)
--------------------

like in check-=

> (check-= 1.0 1.01 0.005 "I fail")
--------------------
FAILURE
name:       check-=
location:   eval:12:0
params:     '(1.0 1.01 0.005)
message:    "I fail"
--------------------
sorawee commented 12 months ago

I think that params is accurate but not ideal. It requires readers to already know beforehand what each position in the list means. If we can come up with an accurate set of field names that can convey the same information, that would be the best.

matteo-daddio commented 12 months ago

What about:

> (check-within (list 6e+23 10.0) (list 6.02e+23 9.8) 0.05)
--------------------
FAILURE
name:         check-within
location:     eval:16:0
first value:  '(6e+23 10.0)
second value: '(6.02e+23 9.8)
epsilon:      0.05
--------------------

> (check-within (hash 'C 18 'F 64) (hash 'C 25 'F 77) 10)
--------------------
FAILURE
name:         check-within
location:     eval:17:0
first value:  '#hash((C . 18) (F . 64))
second value: '#hash((C . 25) (F . 77))
epsilon:      10
--------------------
sorawee commented 12 months ago

They are not just first and second value though. There's an expectation that the first argument is the "actual" value from computation (which could be incorrect), and the second argument is a known-to-be-good value. That is, we would write:

(check-within (f ...) <some-val> <epsilon>)

and not

(check-within <some-val> (f ...) <epsilon>)
matteo-daddio commented 12 months ago

It's not clear how these checks are ment to be used. The RackUnit documentation is quite vague. The examples don't work as you suggested.

> (check-= 1.0 1.01 0.02 "I work")
> (check-= 1.0 1.01 0.005 "I fail")

> (check-within (list 6 10) (list 6.02 9.99) 0.05)
> (check-within (flvector 3.0 4.0 5.0) (flvector 3.01 4.01 5.014) 0.02)
> (check-within (hash 'C 20 'F 68) (hash 'C 25 'F 77) 10)

They work quite the contrary. Maybe the documentation should be improved.

matteo-daddio commented 11 months ago

An improved documentation could be:

(check-= v1 v2 epsilon [message]) → void?
  v1 : number?
  v2 : number?
  epsilon : number?
  message : (or/c string? #f) = #f

Checks that v1 and v2 are numbers within epsilon of one another. Usually the first number is produced by a function, the second number is the expected value and epsilon is the tolerance. The optional message is included in the output if the check fails.

For example, the following check passes:

> (define (golden-ratio) 1.62) ;computes the golden ratio
> (check-= (golden-ratio) 1.618033988749  0.01 "I work")

The following check fails:

> (check-= (golden-ratio) 1.618033988749  0.001 "I fail")

--------------------
FAILURE
name:       check-=
location:   eval:12:0
actual:     1.62
expected:   1.618033988749
tolerance:  0.001 
message:    "I fail"
--------------------

(check-within v1 v2 epsilon [message]) → void?
  v1 : any/c
  v2 : any/c
  epsilon : number?
  message : (or/c string? #f) = #f

Checks that v1 and v2 are equal? to each other, while allowing numbers inside of them to be different by at most epsilon from one another. If (equal? v1 v2) would call equal? on sub-pieces that are numbers, then those numbers are considered "good enough" if they’re within epsilon. This check is similar to check-= except it works on data structures like lists, flvectors and hash tables.

For example, the following checks pass:

> (check-within (list 6.02 9.99) (list 6 10) 0.05)
> (check-within (flvector 3.01 4.01 5.014) (flvector 3.0 4.0 5.0) 0.02)
> (check-within (hash 'C 25 'F 77) (hash 'C 20 'F 68) 10)

And the following checks fail:

> (check-within (list 6.02e+23 9.8) (list 6e+23 10.0) 0.05)

--------------------
FAILURE
name:       check-within
location:   eval:16:0
actual:     '(6.02e+23 9.8)
expected:   '(6e+23 10.0)
tolerance:  0.05
--------------------

> (check-within (hash 'C 18 'F 64) (hash 'C 25 'F 77) 10)

--------------------
FAILURE
name:       check-within
location:   eval:17:0
actual:     '#hash((C . 18) (F . 64))
expected:   '#hash((C . 25) (F . 77))
tolerance:  10
--------------------
sorawee commented 10 months ago

Fixed by #166