rejeep / ert-runner.el

Opinionated Ert testing workflow
85 stars 19 forks source link

Add xfail to dot reporter #22

Closed immerrr closed 9 years ago

immerrr commented 9 years ago

This adds expected failures drawn as x and unexpected successes drawn as X in dot reporter.

rejeep commented 9 years ago

Looks like ert-test-skipped-p isn't available before Emacs 24.3. Can you do something like this (https://github.com/flycheck/flycheck/blob/master/flycheck-ert.el#L41-L62) to make sure it works in older Emacsen!?

swsnr commented 9 years ago

Or just use a dummy predicate on 24.2 and earlier, since you can't get skipped tests on these anyway.

immerrr commented 9 years ago

All green (I have added a simple fboundp check).

rejeep commented 9 years ago

Just to make sure I understand this change. Will ert-test-passed-p/ert-test-failed-p only return t if :expected-result is used? So this change should only effect tests using :expected-result?

immerrr commented 9 years ago

Will ert-test-passed-p/ert-test-failed-p only return t if :expected-result is used?

ert-test-passed-p is a predicate for cl-defstruct that is returned when a test passes, so it will return t for passed tests regardless of :expected-result (which is :passed by default).

So this change should only effect tests using :expected-result?

The output should change only for those tests (and those that are skipped). Internally, most of the cases will use different codepaths. The default branch of cond is left for backward compatibility with other test outcomes (:quit and :aborted-with-non-local-exit).

immerrr commented 9 years ago

Actually, don't merge this just yet, I have found an interesting function ert-char-for-test-result that can replace that function completely. I wonder if it is available in earlier versions...

immerrr commented 9 years ago

Yep, it is!

immerrr commented 9 years ago

It changed the characters for expected failures from xX to fP, though, but that's even better since it will be consistent with ert.

rejeep commented 9 years ago

Merged, thanks!