search

ngauthier / hydra

Distributed testing framework
MIT License
389 stars 63 forks source link

Hydra not failing all the tests #48

Open leenasn opened 13 years ago

leenasn commented 13 years ago

Hi,

I configured hydra for my rails app. All my tests are failing when am running with hydra. It also does not print the reason why it is failing. Couple of things I would like to know:

  1. How to print the details of the failing tests
  2. How to use rspec style reporting for printing the details. In .rspec file, I have --format -d, but seems like it does not have any effect for hydra.

My setup is:

And the following is what I've added in the Rakefile:

# require the hydra codebase
require 'hydra'
# require the hydra rake task helpers
require 'hydra/tasks'

# set up a new hydra testing task named 'hydra:spec' run with "rake hydra:spec"
Hydra::TestTask.new('hydra:spec') do |t|
  # you may or may not need this, depending on how you require
  # spec_helper in your test files:
  require './spec/spec_helper'
  # add all files in the spec directory that end with "_spec.rb"
  t.add_files './spec/models/*.rb'
end

The following the stack trace what I get when I run with --trace option for rake:

Hydra: Not all tests passes /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/hydra-0.23.3/lib/hydra/tasks.rb:120:inblock in define' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:636:in call' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:636:inblock in execute' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:631:in each' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:631:inexecute' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:597:in block in invoke_with_call_chain' /var/lib/hudson/.rvm/rubies/ruby-1.9.2-rc1/lib/ruby/1.9.1/monitor.rb:201:inmon_synchronize' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:590:in invoke_with_call_chain' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:583:ininvoke' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:2051:in invoke_task' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:2029:inblock (2 levels) in top_level' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:2029:in each' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:2029:inblock in top_level' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:2068:in standard_exception_handling' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:2023:intop_level' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:2001:in block in run' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:2068:instandard_exception_handling' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:1998:in run' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/bin/rake:31:in<top (required)>' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/bin/rake:19:in load' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/bin/rake:19:in

'`

Please let me know how to fix the issue.

Thanks, Leena

ngauthier commented 13 years ago

Hey Leena,

Hydra runs Rspec with a special Hydra Formatter so it can parse the results, so you can't get back the pretty output.

From your trace, it looks like the definition of the hydra rake task is what's failing. I am not sure if hydra is Ruby 1.9 compatible, last time I checked it wasn't. Also, Rake 0.9.x seems to have changed a couple of things too, so that could be the issue.

Sorry hydra is not keeping up to date with the latest and greatest libs. Try parallel-test?

-Nick

On Fri, Jun 24, 2011 at 8:38 AM, leenasn < reply@reply.github.com>wrote:

Hi,

I configured hydra for my rails app. All my tests are failing when am running with hydra. It also does not print the reason why it is failing. Couple of things I would like to know:

  1. How to print the details of the failing tests
  2. How to use rspec style reporting for printing the details. In .rspec file, I have --format -d, but seems like it does not have any effect for hydra.

My setup is:

  • Rails 3.0.9
  • RSpec 2.5.0
  • Hydra 0.23.3
  • Ruby 1.9.2 (with rvm)
  • Ubuntu 10.04

And the following is what I've added in the Rakefile:

# require the hydra codebase
require 'hydra'
# require the hydra rake task helpers
require 'hydra/tasks'

# set up a new hydra testing task named 'hydra:spec' run with "rake
hydra:spec"
Hydra::TestTask.new('hydra:spec') do |t|
 # you may or may not need this, depending on how you require
 # spec_helper in your test files:
 require './spec/spec_helper'
 # add all files in the spec directory that end with "_spec.rb"
 t.add_files './spec/models/*.rb'
end

The following the stack trace what I get when I run with --trace option for rake:

Hydra: Not all tests passes /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/hydra-0.23.3/lib/hydra/tasks.rb:120:in block in define' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:636:in call' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:636:in block in execute' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:631:in each' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:631:in execute' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:597:in block in invoke_with_call_chain' /var/lib/hudson/.rvm/rubies/ruby-1.9.2-rc1/lib/ruby/1.9.1/monitor.rb:201:in mon_synchronize' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:590:in invoke_with_call_chain' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:583:in invoke' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:2051:in invoke_task' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:2029:in block (2 levels) in top_level' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:2029:in each' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:2029:in block in top_level' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:2068:in standard_exception_handling' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:2023:in top_level' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:2001:in block in run' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:2068:in standard_exception_handling' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/lib/rake.rb:1998:in run' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/gems/rake-0.8.7/bin/rake:31:in <top (required)>' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/bin/rake:19:in load' /var/lib/hudson/.rvm/gems/ruby-1.9.2-rc1/bin/rake:19:in

'`

Please let me know how to fix the issue.

Thanks, Leena

Reply to this email directly or view it on GitHub: https://github.com/ngauthier/hydra/issues/48

leenasn commented 13 years ago

I tried parallel_tests and it worked fine, thanks for your suggestion.

ghost commented 13 years ago

I am currently in the process of adapting hydra to use multiple formatters for both Cucumber and RSpec, for my company's internal application. Right now, I'm just monkey-patching the definitions to add JUnit formatters to both of them, but soon I'll have time to properly modify hydra.

ngauthier commented 13 years ago

Sounds cool. I look forward to it!

On Wed, Jun 29, 2011 at 9:07 PM, mirell < reply@reply.github.com>wrote:

I am currently in the process of adapting hydra to use multiple formatters for both Cucumber and RSpec, for my company's internal application. Right now, I'm just monkey-patching the definitions to add JUnit formatters to both of them, but soon I'll have time to properly modify hydra.

Reply to this email directly or view it on GitHub: https://github.com/ngauthier/hydra/issues/48#issuecomment-1471936