search

jorgenschaefer / emacs-buttercup

Behavior-Driven Emacs Lisp Testing
GNU General Public License v3.0
360 stars 44 forks source link

spec *-each and function should only set "more serious" state #157

Closed snogge closed 4 years ago

snogge commented 4 years ago

As discussed in #149, any before-each, buttercup-spec-function or after-each should only set the state of the spec if the new state is "more serious" than the specs current state.

This means that pendingshould win over success and failure should win over both pending and success. success should never win over any other state.

All in all we think this will improve the test results.

DarwinAwardWinner commented 4 years ago

I'm not sure if this is the same bug, but using assume in a before-each form skips the remainder of the before-each but then seems to run the body of the test as normal (which then fails because the assumption was false).

In addition, using assume in the body of a test gets overwritten with a passed state if there is an after-each form. I'm pretty sure this example is an instance of this bug.

Also, I don't believe the assume macro is documented anywhere. I'm not even certain whether it's meant to be used in a before-each form, but it seems like it should work there.