search

racket / rackunit

Other
18 stars 32 forks source link

`run-test` falsely claims that is outputs to current-output-port #135

Closed wilbowma closed 3 years ago

wilbowma commented 3 years ago

What version of Racket are you using? e.g., 8.0.0.9 [cs], commit 18435e3a082fbcfdd09d76f06f3b1349a386bab6

What program did you run?

#lang racket
(require
 rackunit
 rackunit/gui
 rackunit/text-ui)

(module+ test
  (check-equal?
   (with-output-to-string
     (thunk
      (run-tests
       (test-suite
        ""
        (check-equal? 5 6)))))
   "0 success(es) 1 failure(s) 0 error(s) 1 test(s) run\n")

  (check-equal?
   (with-output-to-string
     (thunk
      (parameterize ([current-error-port (current-output-port)]
                     [current-output-port (open-output-string)])
        (run-tests
         (test-suite
          ""
          (check-equal? 5 6))))))
   ""))

What should have happened? According to the documentation, run-tests outputs to the current-output-port, so these tests should pass. However, it seems run-tests only outputs to the current-output-port if all tests pass; otherwise, it outputs to current-error-port. This makes sense, I guess, but is at odds with the docs, and I'm not sure which behaviour to believe.

jackfirth commented 3 years ago

Transferred this to the rackunit repository since it seems like a rackunit-specific issue.

I think updating the docs is the right move. A lot of CI systems decide whether or not your CI checks failed based on whether there was any output to the stderr, so rackunit's behavior is probably intended to cooperate with that.

wilbowma commented 3 years ago

My mistake; thanks for the correction

wilbowma commented 3 years ago

Closed by #137