Zeus, Rails4 (beta1) and Test::Unit cause test suite to run twice (or not at all)

[murdoch]

If I run zeus test, then I see:

/home/stephen/.rvm/gems/ruby-2.0.0-p0@murdoch/gems/zeus-0.13.2/lib/zeus/rails.rb:210:in `spec_file?': undefined method `match' for nil:NilClass (NoMethodError)

If I run zeus test test or zeus rake test then all of my tests run twice

zeus test test is much faster than zeus rake test

zeus (0.13.2) rails (4.0.0.beta1) ruby (2.0.0 p0)

Sorry if this is a dup.

[ishe-ua]

i have the same issue. i use rails 3.2.12, minitest-rails and last version of zeus. maybe the problem is in minitest.

[turadg]

Does version 0.13.3 help?

[ishe-ua]

yes, 0.13.3

[ishe-ua]

my output is:

ishe@fuji:~/code/tag-communities$ zeus test test/models/member_test.rb 
Run options: --seed 51413

# Running tests:

....................

Fabulous tests in 2.431165s, 8.2265 tests/s, 9.4605 assertions/s.

20 tests, 23 assertions, 0 failures, 0 errors, 0 skips
Run options: --seed 61954

# Running tests:

....................

Fabulous tests in 1.473757s, 13.5708 tests/s, 15.6064 assertions/s.

20 tests, 23 assertions, 0 failures, 0 errors, 0 skips

[mbhnyc]

I'm having a basically identical issue.

[murdoch]

It's very strange, but for some inexplicable reason the issue stopped occurring sometime last week. I can't replicate it using the latest gem (zeus (0.13.3)) either. Dunno what made it go away, will keep my eyes peeled though.

[fivetwentysix]

I'm getting this issue too. I've got 3 rails apps running outofthebox MiniTest on Rails 4 in Ruby2.0.0-p0

1) Error: UsersControllerTest#test_creating_account: ActionController::UrlGenerationError: No route matches {:code=>"4/BMnDVTNYx67SIgI7LIygjPfKCyOF.IuiJcRQeFy8YgrKXntQAax2iHtnaegI", :redirect_uri=>"http://localhost:3000/oauth/callback/google", :controller=>"users", :action=>"create"} test/controllers/users_controller_test.rb:9:in block (2 levels) in <class:UsersControllerTest>' test/controllers/users_controller_test.rb:8:inblock in class:UsersControllerTest'

1 tests, 0 assertions, 0 failures, 1 errors, 0 skips Run options: --seed 38397

Running tests:

E

Finished tests in 0.004902s, 203.9984 tests/s, 0.0000 assertions/s.

1) Error: UsersControllerTest#test_creating_account: ActionController::UrlGenerationError: No route matches {:code=>"4/BMnDVTNYx67SIgI7LIygjPfKCyOF.IuiJcRQeFy8YgrKXntQAax2iHtnaegI", :redirect_uri=>"http://localhost:3000/oauth/callback/google", :controller=>"users", :action=>"create"} test/controllers/users_controller_test.rb:9:in block (2 levels) in <class:UsersControllerTest>' test/controllers/users_controller_test.rb:8:inblock in class:UsersControllerTest'

2) Error: UsersControllerTest#test_creating_account: ActionController::UrlGenerationError: No route matches {:code=>"4/BMnDVTNYx67SIgI7LIygjPfKCyOF.IuiJcRQeFy8YgrKXntQAax2iHtnaegI", :redirect_uri=>"http://localhost:3000/oauth/callback/google", :controller=>"users", :action=>"create"} test/controllers/users_controller_test.rb:9:in block (2 levels) in <class:UsersControllerTest>' test/controllers/users_controller_test.rb:8:inblock in class:UsersControllerTest'

1 tests, 0 assertions, 0 failures, 2 errors, 0 skips

[Zelnox]

It happens to me too. Rails 3.1.11 + zeus 0.13.3 + minitest 4.6.2

[mbhnyc]

Glad I'm not the only one!

[sevos]

Don't you have minitest/autorun required anywhere?

[mbhnyc]

Not I!

[DavertMik]

+1 for this issue rails 3.2.12, zeus 0.13.3, minitest 4.7.0

[RedaBenh]

+1 for this issue

[soffes]

Same here. Using minitest.

Looked into a similar issue with the m gem. Basically loading the tests will run them and then minitest/autorun will run them before it quits. Never found a good solution to this problem for m.

[mbhnyc]

extra +1 - no one's found a definitive cause for this yet? it's rather urgh inducing. :dash:

[deiga]

I have the same issue with Rails 3.2.12, Rspec 2.13, zeus 0.13.3.

Only difference that zeus test spec ends in a weird error https://gist.github.com/deiga/5349820

It also doesn't use a random seed.

zeus rake spec works normally.

I started investigating this when guard-rspec ran all tests double (because of zeus) when I modified files

[mbhnyc]

FIXED!

I was updating minitest-rails and looking for ways to make my tests more efficient, and fixed up my custom_plan ... looking closely I noticed:

def test_environment bundler_group, load_path
    Bundler.require :test, bundler_group
    require APP_PATH
    $rails_rake_task = 'yup' # lie to skip eager loading
    ::Rails.application.require_environment!
    $rails_rake_task = nil 
    $LOAD_PATH.unshift ".", "./lib", load_path
  end

  def minitest_environment
    test_environment :test, "./test"
    require 'minitest_helper'
  end

Which surely was copied and pasted from somewhere.

Anyway, the key is here: Bundler.require :test, bundler_group which was basically requiring my environment twice. Switching to Bundler.require :test fixed this completely

My New shiny custom_plan:

require 'zeus/rails'

class CustomPlan < Zeus::Rails

  def test_bundle
    ::Rails.env = ENV['RAILS_ENV'] = 'test'
    default_bundle
  end

  def test_environment
    Bundler.require :test
    require APP_PATH
    $rails_rake_task = 'yup' # lie to skip eager loading
    ::Rails.application.require_environment!
    $rails_rake_task = nil
    $LOAD_PATH.unshift ".", "./lib", "./test"
    require 'test_helper'
  end

end

Zeus.plan = CustomPlan.new

[latortuga]

I can replicate a very similar issue to this on Rails 3.2.12, minitest 4.7.3, and zeus 0.13.3.rc2 if I put my test_helper require statements in the wrong order:

# Works great!
require 'rails/test_help'
require 'minitest/rails'

# Double test runs
require 'minitest/rails'
require 'rails/test_help'

Note that this is most likely a PEBKAC-type issue - the minitest-rails wiki advises the former ordering in their upgrading minitest-rails wiki entry.

[diego-aslz]

+1. Same issue here, Rails 3.2.13, Rspec 2.13, Zeus 0.13.

[diego-aslz]

I think I found the solution. I'm using Rspec. Removing either of these lines from my spec_helper solved the problem for me.

    require 'rspec/autorun'
    require 'rspec/autotest'

I found it here.

[yasuoza]

+1. Same issue, rails 4.0.0rc1, zeus 0.13.3, with default rails test framework.

It seems zeus test test runs all tests first and MiniTest::Unit.autorun defined in active_support/testing/autorun.rb runs all tests again.

[yasuoza]

Temporary solution for rails default test framework.

Using zeus custom plan:

require 'zeus/rails'

class CustomPlan < Zeus::Rails

  def test_helper
    require 'minitest/unit'
    MiniTest::Unit.class_variable_set("@@installed_at_exit", true)
    super
  end

end

Zeus.plan = CustomPlan.new

Using only test_helper.rb:

ENV["RAILS_ENV"] ||= "test"
require File.expand_path('../../config/environment', __FILE__)
if ENV.keys.grep(/ZEUS/).any?
  require 'minitest/unit'
  MiniTest::Unit.class_variable_set("@@installed_at_exit", true)
end
require 'rails/test_help'

class ActiveSupport::TestCase
  ActiveRecord::Migration.check_pending!

  # Setup all fixtures in test/fixtures/*.yml for all tests in alphabetical order.
  #
  # Note: You'll currently still have to declare fixtures explicitly in integration tests
  # -- they do not yet inherit this setting
  fixtures :all

  # Add more helper methods to be used by all tests here...
end

[burke]

Fixed, I believe, thanks to @sideshowcoder!

[dfl]

rails (4.0.0), minitest (4.7.5), zeus (0.13.3)

yasuoza's CustomPlan solution, nor solely his test_helper solution, did not work for me. This is the only thing I could get to only run the tests once:

if ENV.keys.grep(/ZEUS/).any?
  require 'minitest/unit'
  MiniTest::Unit.class_variable_set("@@installed_at_exit", true)
end
require "minitest/rails"
require "rails/test_help"

note both the ZEUS env block, and the fact that rails/test_help is required after minitest.

[lscott3]

@dfl your solution worked for me. Thanks.

[rscarvalho]

@dfl thanks for the workaround, that worked for me as well.