fukamachi / prove

Yet another unit testing framework for Common Lisp
218 stars 23 forks source link

Introducing diag in subtest causes subtest to be reported as fail #33

Open fourier opened 7 years ago

fourier commented 7 years ago
(subtest "Diag test"
  (is 1 1)
  (diag "now something else")
  (is #\a #\a))

Output (with (setf prove:default-reporter :tap)):

# Diag test
    ok 1
    # now something else
    ok 2
not ok 13 - Diag test

Whereas:

(subtest "Diag test"
  (is 1 1)
  ;;(diag "now something else")
  (is #\a #\a))

outputs:

# Diag test
    ok 1
    ok 2
ok 13 - Diag test

Here 13 is a number of other tests in the same file.

fourier commented 7 years ago

This I can reproduce with tap report type. I believe probably it is an error somewhere in reporter/tap.lisp:

(defmethod format-report (stream (reporter tap-reporter) (report test-report) &key count)
  (with-slots (description print-error-detail) report
    (format/indent reporter stream
                   "~&~:[not ~;~]ok~:[~;~:* ~D~]~:[~;~:* - ~A~]~%"
                   (or (passed-report-p report)
                       (skipped-report-p report))
                   count
                   description)
    (print-error-report reporter report stream)))

doesn't look like comment-report is taken into consideration.