racket / rackunit

Other
18 stars 34 forks source link

the exception handling in `check-equal?` catches `exn:break?` exceptions and it probably shouldn't #121

Closed rfindler closed 2 months ago

rfindler commented 4 years ago

I think that exn:break? exceptions should not be caught by rackunit's exception handling mechanism. They shouldn't count as a failed test due to an error, they should just be unhandled as a break by the program. Below is an example program (that can be turned into a test case by, say, catching stderr) that, IMO, should behave differently.

#lang racket

(require rackunit)

(define s (make-semaphore))
(define t
  (thread
   (λ ()
     (check-equal? (let ()
                     (semaphore-post s)
                     (semaphore-wait (make-semaphore 0)))
                   5))))
(semaphore-wait s)
(break-thread t)
AlexKnauth commented 4 years ago

I think this is caused by default-check-around's use of (with-handlers ([(λ (_) #t) ....]) ....) in rackunit/private/check line 70 https://github.com/racket/rackunit/blob/ce0ba9f7b1f7bbee4150a246ceb0052272186621/rackunit-lib/rackunit/private/check.rkt#L70

And possibly other with-handlers catch-all clauses in other implementations of check-around functions or test-case-around functions that catch exceptions like this.

rfindler commented 4 years ago

(In case it wasn't clear, I think that should be exn:fail? not the constant true function.)