jorgenschaefer / emacs-buttercup

Behavior-Driven Emacs Lisp Testing
GNU General Public License v3.0
360 stars 44 forks source link

Test failures under Emacs 28 #209

Closed phst closed 2 years ago

phst commented 2 years ago

With a recent Emacs build from the Emacs 28 release branch (commit e3427faf55d7887de1dcba0fd027474ccbb5c214), I get the following errors:

./bin/buttercup -L . tests 
Running 212 specs.

The buttercup-failed signal
  can be raised
  can be raised (140.87ms)

The buttercup-pending signal
  can be raised
  can be raised (0.37ms)

The `expect' form
  with a matcher should translate to the function call with closures
  with a matcher should translate to the function call with closures (107.53ms)
  with no matcher should use `:to-be-truthy' as the matcher
  with no matcher should use `:to-be-truthy' as the matcher (0.75ms)

The `buttercup-expect' function
  with a function as a matcher argument
    should not raise an error if the function returns true
    should not raise an error if the function returns true (0.41ms)
    should raise an error if the function returns false
    should raise an error if the function returns false (0.43ms)
  with a matcher argument
    should not raise an error if the matcher returns true
    should not raise an error if the matcher returns true (0.37ms)
    should raise an error if the matcher returns false
    should raise an error if the matcher returns false (0.40ms)

The `buttercup-fail' function
  should raise a signal with its arguments
  should raise a signal with its arguments (0.46ms)

The `assume' form
  should raise a signal if the condition is nil
  should raise a signal if the condition is nil (0.37ms)
  should show the format if no message is given
  should show the format if no message is given (0.37ms)
  should not raise a signal if the condition is non-nil
  should not raise a signal if the condition is non-nil (0.34ms)

The `buttercup-skip' function
  should raise a signal with its arguments
  should raise a signal with its arguments (0.37ms)

The `buttercup-define-matcher' macro
  should create a matcher usable by apply-matcher
  should create a matcher usable by apply-matcher (0.24ms)

The `buttercup--apply-matcher' function
  should work with functions
  should work with functions (0.35ms)
  should work with matchers
  should work with matchers (0.27ms)
  should fail if the matcher is not defined
  should fail if the matcher is not defined (0.36ms)

The :buttercup-define-matcher-for-unary-function helper
  should not modify match data
  should not modify match data (0.40ms)

The :buttercup-define-matcher-for-binary-function helper
  should not modify match data
  should not modify match data (0.40ms)

The `buttercup-suite-add-child' function
  should add an element at the end of the list and return it
  should add an element at the end of the list and return it (0.50ms)
  should add an element even if the list is empty and return it
  should add an element even if the list is empty and return it (0.43ms)
  should add the parent to the child
  should add the parent to the child (0.26ms)

The `buttercup-suite-parents' function
  should return the list of parents for a suite
  should return the list of parents for a suite (0.30ms)

The `buttercup-spec-parents' function
  should return the list of parents for a spec
  should return the list of parents for a spec (0.29ms)

The `buttercup-suites-total-specs-defined' function
  should return the number of specs in a list of suites
  should return the number of specs in a list of suites (0.24ms)

The `buttercup-suites-total-specs-pending' function
  should return the number of pending specs in a list of suites
  should return the number of pending specs in a list of suites (0.30ms)
  should also count skipped specs
  should also count skipped specs (0.38ms)

The `buttercup-suites-total-specs-failed' function
  should return the number of failed specs in a list of suites
  should return the number of failed specs in a list of suites (0.26ms)

The `buttercup-suite-full-name' function
  should return the full name of a suite without parents
  should return the full name of a suite without parents (0.30ms)
  should return the full name of a suite with parents
  should return the full name of a suite with parents (0.25ms)

The `buttercup-spec-full-name' function
  should return the full name of a spec without parents
  should return the full name of a spec without parents (0.26ms)
  should return the full name of a spec with parents
  should return the full name of a spec with parents (0.27ms)

The `buttercup-elapsed-time' function
  should report elapsed time for suites
  should report elapsed time for suites (0.35ms)
  should report elapsed time for specs
  should report elapsed time for specs (0.30ms)

The `buttercup--run-suite' function
  should set start and end time of the suite
  should set start and end time of the suite (0.54ms)

The `buttercup--run-spec' function
  should set start and end time of the spec
  should set start and end time of the spec (0.65ms)
  should not overwrite pending status with `after-each' results
  should not overwrite pending status with `after-each' results (0.48ms)
  should set status to pending
    for assume in `before-each'
    for assume in `before-each' (0.77ms)
    for assume in spec
    for assume in spec (0.44ms)
    for assume in `after-each'
    for assume in `after-each' (0.94ms)

The `describe' macro
  should expand to a simple call to the buttercup-describe function
  should expand to a simple call to the buttercup-describe function (0.42ms)
  should support the :var argument
  should support the :var argument (0.44ms)
  should support the :var* argument
  should support the :var* argument (0.40ms)

The `buttercup-describe' function
  should run the enclosing body
  should run the enclosing body (0.21ms)
  should set the `buttercup-suites' variable
  should set the `buttercup-suites' variable (0.26ms)
  should add child suites when called nested
  should add child suites when called nested (0.43ms)

The `it' macro
  should expand to a call to the `buttercup-it' function
  should expand to a call to the `buttercup-it' function (0.26ms)
  without argument should expand to xit.
  without argument should expand to xit. (0.25ms)

The `buttercup-it' function
  should fail if not called from within a describe form
  should fail if not called from within a describe form (0.37ms)
  should add a spec to the current suite and return the spec
  should add a spec to the current suite and return the spec (0.64ms)

The `before-each' macro
  expands to a function call
  expands to a function call (0.28ms)

The `buttercup-before-each' function
  adds its argument to the before-each list of the current suite
  adds its argument to the before-each list of the current suite (0.26ms)

The `after-each' macro
  expands to a function call
  expands to a function call (0.26ms)

The `buttercup-after-each' function
  adds its argument to the after-each list of the current suite
  adds its argument to the after-each list of the current suite (0.27ms)

The `before-all' macro
  expands to a function call
  expands to a function call (0.31ms)

The `buttercup-before-all' function
  adds its argument to the before-all list of the current suite
  adds its argument to the before-all list of the current suite (0.27ms)

The `after-all' macro
  expands to a function call
  expands to a function call (0.26ms)

The `buttercup-after-all' function
  adds its argument to the after-all list of the current suite
  adds its argument to the after-all list of the current suite (0.26ms)

The `xdescribe' macro
  expands directly to a function call
  expands directly to a function call (0.32ms)
  changes contained it-specs to pending specs
  changes contained it-specs to pending specs (0.46ms)
  should add a pending suite
  should add a pending suite (0.32ms)

The `xit' macro
  expands directly to a function call
  expands directly to a function call (0.28ms)

The `buttercup-xit' function
  should be a no-op
  should be a no-op (0.23ms)
  should add a function that raises a pending signal
  should add a function that raises a pending signal (0.38ms)
  should mark the suite as pending
  should mark the suite as pending (0.26ms)
  should set the failure description to PENDING
  should set the failure description to PENDING (0.62ms)

The Spy 
  `spy-on' function
    replaces a symbol's function slot
    replaces a symbol's function slot (0.31ms)
    restores the old value after a spec run
    restores the old value after a spec run (0.30ms)
    allows a spied-on command to be executed as a command
    allows a spied-on command to be executed as a command (0.71ms)
    can spy on autoloaded functions
    can spy on autoloaded functions (59.86ms)
    can spy on non-existing functions
    can spy on non-existing functions (0.33ms)
    only accepts ARG for keywords that use it
    only accepts ARG for keywords that use it (4.55ms)
    will signal en error if
      used in before-all
      used in before-all (1.17ms)
      used directly in describe
      used directly in describe (1.66ms)
  :to-have-been-called matcher
    returns false if the spy was not called
    returns false if the spy was not called (1.20ms)
    returns true if the spy was called at all
    returns true if the spy was called at all (0.80ms)
  :to-have-been-called-with matcher
    returns false if the spy was not called at all
    returns false if the spy was not called at all (0.82ms)
    returns false if the spy was called with different arguments
    returns false if the spy was called with different arguments (1.38ms)
    returns true if the spy was called with those arguments
    returns true if the spy was called with those arguments (1.16ms)
  :to-have-been-called-times matcher
    returns error if the spy was called less than expected
    returns error if the spy was called less than expected (0.91ms)
    returns error if the spy was called more than expected
    returns error if the spy was called more than expected (0.92ms)
    returns true if the spy was called the expected number of times
    returns true if the spy was called the expected number of times (1.02ms)
    use plural words in error message
    use plural words in error message (1.39ms)
    use singular expected word in error message
    use singular expected word in error message (1.33ms)
    use singular actual word in error message
    use singular actual word in error message (1.37ms)
  :and-call-through keyword functionality
    tracks calls to the function
    tracks calls to the function (0.31ms)
    passes the arguments to the original function
    passes the arguments to the original function (0.82ms)
  :and-return-value keyword functionality
    tracks calls to the function
    tracks calls to the function (0.29ms)
    returns the specified value
    returns the specified value (1.12ms)
    works with strings
    works with strings (1.02ms)
    works with vectors
    works with vectors (0.92ms)
    works with symbols
    works with symbols (0.55ms)
    works with conses
    works with conses (0.54ms)
    works with lists
    works with lists (0.51ms)
    works with alists
    works with alists (0.54ms)
  :and-call-fake keyword functionality
    tracks calls to the function
    tracks calls to the function (0.20ms)
    returns the specified value
    returns the specified value (0.44ms)
  :and-throw-error keyword functionality
    throws an error when called
    throws an error when called (0.63ms)
  error-recording functionality
    records the function as called even if it throws an error
    records the function as called even if it throws an error (117.20ms)
    counts both successful calls and calls that threw errors
    counts both successful calls and calls that threw errors (0.60ms)
    records args to the function whether it throw an error or not
    records args to the function whether it throw an error or not (0.73ms)
    records the signal thrown by a call to the function
    records the signal thrown by a call to the function (0.81ms)

The batch reporters
  on the buttercup-started event
    should print the number of specs
    should print the number of specs (0.52ms)
    should color-print the number of specs with the default color
    should color-print the number of specs with the default color (0.45ms)
    should print the number of skipped specs
    should print the number of skipped specs (0.50ms)
    should color-print the number of skipped specs with the default color
    should color-print the number of skipped specs with the default color (0.46ms)
  on the suite-started event
    should emit an indented suite description
    should emit an indented suite description (0.38ms)
    should color-print an indented suite description with the default color
    should color-print an indented suite description with the default color (0.38ms)
  on the spec-started event
    should emit an indented spec description
    should emit an indented spec description (0.36ms)
    should color-print an indented spec description with the default color
    should color-print an indented spec description with the default color (0.37ms)
  on the spec-done event
    for a passed spec
      should print no status tag
      should print no status tag (0.45ms)
      should color-print the description in green and no status tag
      should color-print the description in green and no status tag  Expected `(buttercup-output)' to be `equal-including-properties' to `#("    spec (0.00ms)
" 0 8 (font-lock-face (:foreground "green3")))', but instead it was `#("    spec (0.00ms)
" 0 8 (font-lock-face (:foreground "green3")))'. (0.73ms)
      should print multiline specs cleanly
      should print multiline specs cleanly (0.52ms)
      should color-print multiline specs cleanly
      should color-print multiline specs cleanly  Expected `(buttercup-output)' to be `equal-including-properties' to `#("    one
two
   three (0.00ms)
" 0 20 (font-lock-face (:foreground "green3")))', but instead it was `#("    one
two
   three (0.00ms)
" 0 8 (font-lock-face (:foreground "green3")) 8 15 (font-lock-face (:foreground "green3")) 15 20 (font-lock-face (:foreground "green3")))'. (0.75ms)
    for a failed spec
      should say FAILED
      should say FAILED (0.51ms)
      should color-print the description in red and say FAILED
      should color-print the description in red and say FAILED  Expected `(buttercup-output)' to be `equal-including-properties' to `#("    spec  FAILED (0.00ms)
" 0 16 (font-lock-face (:foreground "red3")))', but instead it was `#("    spec  FAILED (0.00ms)
" 0 8 (font-lock-face (:foreground "red3")) 8 16 (font-lock-face (:foreground "red3")))'. (0.81ms)
    for a pending spec
      should output the failure-description
      should output the failure-description (0.52ms)
      should color-print the description and failure-description in yellow
      should color-print the description and failure-description in yellow  Expected `(buttercup-output)' to be `equal-including-properties' to `#("    spec  DESCRIPTION (0.01ms)
" 0 21 (font-lock-face (:foreground "yellow3")))', but instead it was `#("    spec  DESCRIPTION (0.01ms)
" 0 8 (font-lock-face (:foreground "yellow3")) 8 21 (font-lock-face (:foreground "yellow3")))'. (0.83ms)
    should throw an error for an unknown spec status
      for plain output
      for plain output (0.40ms)
      for colored output
      for colored output (0.41ms)
  on the suite-done event
    should emit a newline at the end of a top-level suite
    should emit a newline at the end of a top-level suite (0.37ms)
    should color-print a newline at the end of a top-level suite
    should color-print a newline at the end of a top-level suite (0.37ms)
    should not emit anything at the end of other suites
    should not emit anything at the end of other suites (0.31ms)
    should not color-print anything at the end of other suites
    should not color-print anything at the end of other suites (0.32ms)
  on the buttercup-done event
    should print a summary of run and failing specs
    should print a summary of run and failing specs (0.53ms)
    should color-print `0 failed' specs in green
    should color-print `0 failed' specs in green  Expected `(substring (buttercup-output) 0 (length "Ran 10 specs, 0 failed, in"))' to be `equal-including-properties' to `#("Ran 10 specs, 0 failed, in" 14 22 (font-lock-face (:foreground "green3")))', but instead it was `#("Ran 10 specs, 0 failed, in" 14 22 (font-lock-face (:foreground "green3")))'. (0.93ms)
    should color-print `X failed' specs in red
    should color-print `X failed' specs in red  Expected `(substring (buttercup-output) 0 (length "Ran 10 specs, 6 failed, in"))' to be `equal-including-properties' to `#("Ran 10 specs, 6 failed, in" 14 22 (font-lock-face (:foreground "red3")))', but instead it was `#("Ran 10 specs, 6 failed, in" 14 22 (font-lock-face (:foreground "red3")))'. (0.94ms)
    should print a summary separating run and pending specs
    should print a summary separating run and pending specs (0.54ms)
    should color-print pending spec count in default color
    should color-print pending spec count in default color (0.80ms)
    should not raise any error even if a spec failed
    should not raise any error even if a spec failed (0.56ms)
  on an unknown event
    should raise an error
    should raise an error (0.40ms)

Backtraces
  should be printed for each failed spec
  should be printed for each failed spec (2.65ms)
  with style
    `crop' should print truncated lines
    `crop' should print truncated lines (5.26ms)
    `full' should print full lines
    `full' should print full lines (2.27ms)
    `pretty' should pretty-print frames
    `pretty' should pretty-print frames (4.33ms)
    `omit' should print nothing
    `omit' should print nothing (0.61ms)
  should signal an error for unknown styles
  should signal an error for unknown styles (0.46ms)
  should generate correct backtrace for
    no matcher
    no matcher (103.86ms)
    :to-be-truthy
    :to-be-truthy (1.62ms)
    :not :to-be-truthy
    :not :to-be-truthy (1.55ms)
    :to-be
    :to-be (1.50ms)
    :not :to-be
    :not :to-be (1.48ms)
    :to-equal
    :to-equal (1.44ms)
    :not :to-equal
    :not :to-equal (1.44ms)
    :to-have-same-items-as
    :to-have-same-items-as (1.39ms)
    :not :to-have-same-items-as
    :not :to-have-same-items-as (1.36ms)
    :to-match
    :to-match (1.34ms)
    :not :to-match
    :not :to-match (1.33ms)
    :to-be-in
    :to-be-in (1.33ms)
    :not :to-be-in
    :not :to-be-in (1.36ms)
    :to-contain
    :to-contain (1.33ms)
    :not :to-contain
    :not :to-contain (1.37ms)
    :to-be-less-than
    :to-be-less-than (1.36ms)
    :not :to-be-less-than
    :not :to-be-less-than (1.35ms)
    :to-be-greater-than
    :to-be-greater-than (1.35ms)
    :not :to-be-greater-than
    :not :to-be-greater-than (1.38ms)
    :to-be-weakly-less-than
    :to-be-weakly-less-than (1.35ms)
    :not :to-be-weakly-less-than
    :not :to-be-weakly-less-than (1.38ms)
    :to-be-weakly-greater-than
    :to-be-weakly-greater-than (1.37ms)
    :not :to-be-weakly-greater-than
    :not :to-be-weakly-greater-than (1.37ms)
    :to-be-close-to
    :to-be-close-to (1.38ms)
    :not :to-be-close-to
    :not :to-be-close-to (1.39ms)
    :to-have-been-called
    :to-have-been-called (1.36ms)
    :not :to-have-been-called
    :not :to-have-been-called (1.39ms)
    :to-have-been-called-with
    :to-have-been-called-with (109.22ms)
    :not :to-have-been-called-with
    :not :to-have-been-called-with (1.68ms)
    :to-have-been-called-times
    :to-have-been-called-times (1.63ms)
    :not :to-have-been-called-times
    :not :to-have-been-called-times (1.58ms)

When using quiet specs in the batch reporter
  should print nothing if all specs are quiet
  should print nothing if all specs are quiet (1.29ms)
  should print the containing suites for non-quiet specs
  should print the containing suites for non-quiet specs (1.86ms)
  should quiet all of the given spec statuses
  should quiet all of the given spec statuses (1.07ms)
  should handle `skipped' virtual status in quiet list
  should handle `skipped' virtual status in quiet list (2.30ms)
  should handle `disabled' virtual status in quiet list
  should handle `disabled' virtual status in quiet list (2.20ms)

The `buttercup-run' function
  should signal an error if no suites are defined
  should signal an error if no suites are defined (0.42ms)
  should return :no-suites for no suites and noerror
  should return :no-suites for no suites and noerror (0.27ms)
  should raise an error if at least one spec failed
  should raise an error if at least one spec failed (0.41ms)
  should return nil for failing specs and noerror
  should return nil for failing specs and noerror (0.23ms)
  should return t for passing specs
  should return t for passing specs (0.36ms)
  should call the reporter twice with events buttercup-started and -done
  should call the reporter twice with events buttercup-started and -done (0.46ms)
  should call `buttercup--run-suite' once per suite
  should call `buttercup--run-suite' once per suite (0.44ms)

The `buttercup--print' function
  should send a formatted string to the terminal
  should send a formatted string to the terminal (0.11ms)

The `buttercup-mark-skipped' function
  should do nothing with a reversed match-all pattern
  should do nothing with a reversed match-all pattern (2.30ms)
  should mark all specs as pending with a reversed match none pattern
  should mark all specs as pending with a reversed match none pattern (0.74ms)
  should handle multiple patterns
  should handle multiple patterns (0.72ms)
  should support predicates
  should support predicates (0.82ms)
  should support reversed predicates
  should support reversed predicates (0.82ms)
  should signal an error for invalid matchers
  should signal an error for invalid matchers (0.81ms)

Buttercup's ERT compatibility wrapper
  should convert `ert-test-failed' into `buttercup-failed'
  should convert `ert-test-failed' into `buttercup-failed' (0.42ms)
  should convert `ert-test-skipped' into `buttercup-pending'
  should convert `ert-test-skipped' into `buttercup-pending' (0.38ms)

`buttercup-run-discover' should
  parse command line arguments
    ignoring `--'
    ignoring `--' (0.28ms)
    requiring an extra argument for `--traceback'
    requiring an extra argument for `--traceback' (0.42ms)
    checking `--traceback' argument for validity
    checking `--traceback' argument for validity (0.45ms)
    setting `buttercup-stack-frame-style' from `--traceback' arg
    setting `buttercup-stack-frame-style' from `--traceback' arg (0.50ms)
    requiring an extra argument for `--pattern' or `-p'
    requiring an extra argument for `--pattern' or `-p' (0.72ms)
    collecting `--pattern' and `-p' args and send to `buttercup-mark-skipped'
    collecting `--pattern' and `-p' args and send to `buttercup-mark-skipped' (0.51ms)
    clearing `buttercup-color' if `--no-color' is given
    clearing `buttercup-color' if `--no-color' is given (0.80ms)
    adding `skipped' and `disabled' to quiet statuses if `--no-skip' is given
    adding `skipped' and `disabled' to quiet statuses if `--no-skip' is given (0.62ms)
    adding `pending' and `passed' to quiet statuses if `--only-error' is given
    adding `pending' and `passed' to quiet statuses if `--only-error' is given (0.63ms)
    calling `buttercup-error-on-stale-elc' if `--stale-file-error' is given
    calling `buttercup-error-on-stale-elc' if `--stale-file-error' is given (0.32ms)
    search any unknown args for test files
    search any unknown args for test files (0.44ms)
  find and load files
    named test-*.el and *-tests?.el but no other files
    named test-*.el and *-tests?.el but no other files (7.74ms)
    only in given directories
    only in given directories (4.93ms)

butter-minor-mode
  should fontify `describe' special form
  should fontify `describe' special form (0.98ms)
  should fontify `it' special form
  should fontify `it' special form (107.13ms)
  should add special forms to `imenu'
  should add special forms to `imenu' (1.87ms)

For stale `elc' file checks
  `buttercup-check-for-stale-elc'
    should do nothing for `el' files
    should do nothing for `el' files (0.58ms)
    should signal error when `elc' is older than `el'
    should signal error when `elc' is older than `el' (0.66ms)
    should not signal error when `elc' is newer than `el'
    should not signal error when `elc' is newer than `el' (0.62ms)
    should do nothing if the `el' file does not exist
    should do nothing if the `el' file does not exist (0.49ms)
  `buttercup-error-on-stale-elc'
    should activate with no argument
    should activate with no argument (0.29ms)
    should deactivate with almost any argument
    should deactivate with almost any argument (0.27ms)
    should toggle when given `toggle' as argument
    should toggle when given `toggle' as argument (0.44ms)

========================================
The batch reporters on the spec-done event for a passed spec should color-print the description in green and no status tag

Traceback (most recent call last):
  buttercup-fail("%s" "Expected `(buttercup-output)' to be `equal-including-...
  signal(buttercup-failed "Expected `(buttercup-output)' to be `equal-includ...
FAILED: Expected `(buttercup-output)' to be `equal-including-properties' to `#("    spec (0.00ms)
" 0 8 (font-lock-face (:foreground "green3")))', but instead it was `#("    spec (0.00ms)
" 0 8 (font-lock-face (:foreground "green3")))'.

========================================
The batch reporters on the spec-done event for a passed spec should color-print multiline specs cleanly

Traceback (most recent call last):
  buttercup-fail("%s" "Expected `(buttercup-output)' to be `equal-including-...
  signal(buttercup-failed "Expected `(buttercup-output)' to be `equal-includ...
FAILED: Expected `(buttercup-output)' to be `equal-including-properties' to `#("    one
two
   three (0.00ms)
" 0 20 (font-lock-face (:foreground "green3")))', but instead it was `#("    one
two
   three (0.00ms)
" 0 8 (font-lock-face (:foreground "green3")) 8 15 (font-lock-face (:foreground "green3")) 15 20 (font-lock-face (:foreground "green3")))'.

========================================
The batch reporters on the spec-done event for a failed spec should color-print the description in red and say FAILED

Traceback (most recent call last):
  buttercup-fail("%s" "Expected `(buttercup-output)' to be `equal-including-...
  signal(buttercup-failed "Expected `(buttercup-output)' to be `equal-includ...
FAILED: Expected `(buttercup-output)' to be `equal-including-properties' to `#("    spec  FAILED (0.00ms)
" 0 16 (font-lock-face (:foreground "red3")))', but instead it was `#("    spec  FAILED (0.00ms)
" 0 8 (font-lock-face (:foreground "red3")) 8 16 (font-lock-face (:foreground "red3")))'.

========================================
The batch reporters on the spec-done event for a pending spec should color-print the description and failure-description in yellow

Traceback (most recent call last):
  buttercup-fail("%s" "Expected `(buttercup-output)' to be `equal-including-...
  signal(buttercup-failed "Expected `(buttercup-output)' to be `equal-includ...
FAILED: Expected `(buttercup-output)' to be `equal-including-properties' to `#("    spec  DESCRIPTION (0.01ms)
" 0 21 (font-lock-face (:foreground "yellow3")))', but instead it was `#("    spec  DESCRIPTION (0.01ms)
" 0 8 (font-lock-face (:foreground "yellow3")) 8 21 (font-lock-face (:foreground "yellow3")))'.

========================================
The batch reporters on the buttercup-done event should color-print `0 failed' specs in green

Traceback (most recent call last):
  buttercup-fail("%s" "Expected `(substring (buttercup-output) 0 (length \"R...
  signal(buttercup-failed "Expected `(substring (buttercup-output) 0 (length...
FAILED: Expected `(substring (buttercup-output) 0 (length "Ran 10 specs, 0 failed, in"))' to be `equal-including-properties' to `#("Ran 10 specs, 0 failed, in" 14 22 (font-lock-face (:foreground "green3")))', but instead it was `#("Ran 10 specs, 0 failed, in" 14 22 (font-lock-face (:foreground "green3")))'.

========================================
The batch reporters on the buttercup-done event should color-print `X failed' specs in red

Traceback (most recent call last):
  buttercup-fail("%s" "Expected `(substring (buttercup-output) 0 (length \"R...
  signal(buttercup-failed "Expected `(substring (buttercup-output) 0 (length...
FAILED: Expected `(substring (buttercup-output) 0 (length "Ran 10 specs, 6 failed, in"))' to be `equal-including-properties' to `#("Ran 10 specs, 6 failed, in" 14 22 (font-lock-face (:foreground "red3")))', but instead it was `#("Ran 10 specs, 6 failed, in" 14 22 (font-lock-face (:foreground "red3")))'.

Ran 212 specs, 6 failed, in 148.62ms.
Debugger entered--Lisp error: (error "")
  signal(error (""))
  error("")
  buttercup--run-suites((... ... ... ... ... ... ... ... ... ... ... ... ... ... ...) nil)
  buttercup-run()
  buttercup-run-discover()
  command-line-1(("-L" "." "-l" "buttercup" "-f" "buttercup-run-discover" "--" "tests"))
  command-line()
  normal-top-level()

make: *** [Makefile:13: test-buttercup] Error 255