Stop on Failing Example?

[andyl]

Is there a way to make Teaspoon stop execution immediately after a failing example?

I am using the option fail_fast = true, but that seems to prevent the execution of follow-on test suites. I'd like to be able to stop execution immediately after a single failing example. Is this possible?

[jejacks0n]

That's what it does. It exits with a non zero exit code, which is the standard, and so may cause rake to not execute the next task. That's pretty much how every example of a test runner seems to work.

---

Jeremy Jackson

On Apr 1, 2015, at 9:50 AM, andyl notifications@github.com wrote:

Is there a way to make Teaspoon stop execution immediately after a failing example?

I am using the option fail_fast = true, but that seems to prevent the execution of follow-on test suites. I'd like to be able to stop execution immediately after a single failing example. Is this possible?

— Reply to this email directly or view it on GitHub.

[andyl]

Hmm ok. I've attached a screenshot of my console output which shows that it keeps running after the first failing spec. I've tried config.fail_fast=true in the config file, and I've tried using -F and --fail-fast=true` on the command line. No luck. In fact it is not even preventing follow-on suites from running.

I'm using Jasmine 1.3.1. Could that be the issue?? (also using: teaspoon 0.9.1, ubuntu 14.04, guard 2.6.1, spring 1.1.3, ruby 2.2.1)

[jejacks0n]

Hmm.. Guard. I don't know if that's supported in guard, you didn't specify that. =)

[andyl]

Well - it seems to behave the same way, whether run from Guard or from the command line.

I have come up with a hack that solves my problem - I filter the teaspoon output. FYI here is the script...

#!/usr/bin/env ruby

# This script removes unwanted teaspoon text
# Read from stdin: `teaspoon | filter_teaspoon_output`

SHOW_PATTERNS = [      # switch to 'show' mode when a pattern is detected
  "^Teaspoon",
  "^Finished"
]

HIDE_PATTERNS = [      # skip printing if these patterns are detected
  "^Starting" , 
  "^Puma"     , 
  "^\\*"      , 
  "^  2\\)"   ,
  "^Failed examples:"
]

def set_mode(string, mode)
  return :show if SHOW_PATTERNS.find {|reg| string =~ Regexp.new(reg)} 
  return :hide if HIDE_PATTERNS.find {|reg| string =~ Regexp.new(reg)} 
  mode
end

buffer = ""
mode   = :show

while buffer = STDIN.gets do
  mode = set_mode(buffer, mode)
  print buffer if mode == :show
  STDOUT.flush
  buffer = ""
end

[jejacks0n]

That's interesting. You seem to know plenty of ruby, so would do you want to take a look and see if you can submit a proposed functional change or write a spec that exhibits the improper behavior? That'd be nice if you can help point me in the right direction -- minus guard.

[mattscilipoti]

You may be looking to utilize a guard group. I put rspec and teaspoon in a group with :halt_on_fail. If either fails, it doesn't continue.

group :specs, halt_on_fail: true do