Extraneous output using TestTask

[trans]

When using the TestTask, when a test fails or errors, the end of the output is:

rake aborted!
Command failed with status (1): [ruby -I"lib:test" -I ... ]

Is it possible to suppress this (and still get the proper exit code, i.e. 1)? I noticed Rails::TestTask manages it to do it, but I haven't figure out how.

[drbrain]

This output comes from Rake::FileUtils#sh:

$ cat test.rf 
require 'rake/testtask'

task :default do
  sh 'exit 2'
end

$ rake -f test.rf
exit 2
rake aborted!
Command failed with status (2): [exit 2...]
/Users/drbrain/Work/git/rake/test.rf:4:in `block in <top (required)>'
Tasks: TOP => default
(See full trace by running task with --trace)

There is no way to suppress the output. I don't see the value in suppressing the output for Rake::TestTask as it can provide valuable information for figuring out why your test failed (syntax error vs missing require vs test failure).

You can write your own test runner that runs the tests in-process. Rake::TestTask doesn't operate this way because it is safer and more consistent to run tests in their own process.

There is no way to retrieve the exit code from the internal task. Providing this would encourage using rake in inappropriate ways (see #257).

[ddarbyson]

@trans did you ever figure out a way to suppress the rake aborted! ... backtrace?

[javierjulio]

@ddarbyson @trans I have. Rails does this by using a TestTask sub class. The code in the define method seems to prevent the output but don't understand how. Below is a stripped down Rakefile (its all I need for a non Rails project) that leaves out the rake aborted!... output. Hope it helps.

require 'rake/testtask'

module Rails
  class TestTask < Rake::TestTask

    def initialize(name = :test)
      super
      @libs << "test" # lib *and* test seem like a better default
    end

    def define
      task @name do
        libs = @libs - $LOAD_PATH
        $LOAD_PATH.unshift(*libs)
        file_list.each { |fl|
          FileList[fl].to_a.each { |f| require File.expand_path f }
        }
      end
    end
  end

end

task default: :test

task :test do
  Rake::Task['test:run'].invoke
end

namespace :test do
  Rails::TestTask.new(:run) do |t|
    t.pattern = "test/**/*_test.rb"
  end
end

Reference: https://github.com/rails/rails/blob/master/railties/lib/rails/test_unit/sub_test_task.rb

[trans]

@javierjulio Yes, something like that can work -- in most cases. The problem is that the tests are running in the Rake process, so you have all the Rake dependencies loaded while the tests are running. As @drbrain pointed out, it is "safer and more consistent to run tests in their own process". So there's the rub.

@dbrian, the reason that it is useful to suppress this output is for structured test output. For example, I have custom output format that produces JSON, designed to be fed into another program. Not being able to suppress extraneous output means the JSON becomes malformed. That's why this issue arose. It occurs to me that there might be a simple solution. Can that information output to stderr instead of stdout? That should do the trick. And actually it seems like it might be a good idea regardless of this issue.

[ddarbyson]

Good call @trans, sending this to stderr sounds like better handling...

[damphyr]

What I usually do in cases like this is forego use of sh() altogether.

Using systemu I'm capturing stdout and stderr separately. Usually I suppress all output when things are OK and print out everything in case of an error. But since the streams are separate the choice is yours.

Also, to suppress all rake output if it is really, really important, you could monkey patch exception handling

module Rake
  class Application
    def standard_exception_handling
      begin
        yield
      rescue SystemExit => ex
        # Exit silently with current status
        raise 
      rescue OptionParser::InvalidOption => ex
         # Invalid option. Exit silently, option parser has already printed error message.
         exit(false)
      rescue Exception => ex
        # Internal failure. Exit with error message and stack trace.
        $stderr.puts "Exception: #{ex.message}"
        $stderr.puts ex.backtrace.join("\n")
        exit(false)
      end
    end
  end
end