`buttercup-run` exits with a signal that cannot be caught

[doublep]

The function basically looks like this:

(defun buttercup-run ()
  ...
  (when (> (buttercup-suites-total-specs-failed buttercup-suites) 0)
    (error "")))

The problem is that this error cannot be caught. As soon as you try to put the call into a condition-case form:

(condition-case nil
    (buttercupt-run)
  (error ...))

you will catch also buttercup-failed signals, so you will effectively abort the test run early. It seems that buttercup-failed is transformed into a tag throw by buttercup--debugger which is not run if there is a matching condition-case clause up the stack.

Proposal: create a variable buttercup-failure-exit with initial value (lambda () (error "")) and funcall it in buttercup-run when appropriate. Then default behavior will not change, but users would be able to get rid of the error if needed.

[snogge]

I've pushed a suggested fix as seen above, what do you think?

[doublep]

Looks fine. The important thing for me is that I can avoid uncatchable magic error and still tell if it was successful or not.