Shinmera / parachute

An extensible and cross-compatible testing framework.
https://shinmera.github.io/parachute
zlib License
94 stars 9 forks source link

:repoort 'plain for IS-VALUES errors on errors #23

Closed 3b closed 3 years ago

3b commented 3 years ago
(define-test foo 
  (is-values (error "x")
    (eql 1)))
(test 'foo)

->

The value
  #<SIMPLE-ERROR "x" {10323EFCF3}>
is not of type
  LIST
   [Condition of type TYPE-ERROR]

Backtrace:
  0: ((:METHOD FORMAT-RESULT (MULTIPLE-VALUE-COMPARISON-RESULT (EQL :EXTENSIVE))) #<MULTIPLE-VALUE-COMPARISON-RESULT :FAILED (is-values (error "x") (eql 1))> #<unused argument>) [fast-method]
  1: ((:METHOD SUMMARIZE (PLAIN)) #<PLAIN 2, FAILED results>) [fast-method]
  2: (TEST FOO)

with IS,

   (is eql 'x (error "x"))

correctly (if somewhat confusingly) reports

The test form   (error "x")
evaluated to    x
when            x
was expected to be equal under EQL.
Shinmera commented 3 years ago

I also made the report a bit clearer for conditions. It's still not ideal, but currently there's no way to distinguish whether a test errored out or just had an unexpected value, so the report display can't do any better.