search

sconover / wrong

Wrong provides a general assert method that takes a predicate block. Assertion failure messages are rich in detail.
MIT License
434 stars 31 forks source link

Test::Unit assertion count does not include wrong assertions #14

Open bjhess opened 12 years ago

bjhess commented 12 years ago

I have a test file that is comprised entirely of wrong assertions (wrong_wrong.rb):

require 'active_support'
require 'wrong'

class WrongWrongTest < ActiveSupport::TestCase
  include Wrong

  def test_if_wrong_is_right
    assert{ true }
  end

  def test_if_wrong_is_right_about_being_wrong
    assert{ false }
  end
end

I run the tests:

~/tmp $ testrb wrong_wrong.rb
Loaded suite WrongWrongTest
Started
E

Error:
test_if_wrong_is_right_about_being_wrong(WrongWrongTest):
Wrong::Assert::AssertionFailedError: Expected false
    /Users/barry/.rvm/gems/ree-1.8.7-2010.02/gems/wrong-0.6.0/lib/wrong/assert.rb:72:in `aver'
    /Users/barry/.rvm/gems/ree-1.8.7-2010.02/gems/wrong-0.6.0/lib/wrong/assert.rb:34:in `assert'
    /Users/barry/tmp/wrong_wrong.rb:12:in `test_if_wrong_is_right_about_being_wrong'
    /Users/barry/.rvm/gems/ree-1.8.7-2010.02/gems/activesupport-3.1.0/lib/active_support/testing/setup_and_teardown.rb:67:in `__send__'
    /Users/barry/.rvm/gems/ree-1.8.7-2010.02/gems/activesupport-3.1.0/lib/active_support/testing/setup_and_teardown.rb:67:in `run'
    /Users/barry/.rvm/gems/ree-1.8.7-2010.02/gems/activesupport-3.1.0/lib/active_support/callbacks.rb:390:in `_run_setup_callbacks'
    /Users/barry/.rvm/gems/ree-1.8.7-2010.02/gems/activesupport-3.1.0/lib/active_support/callbacks.rb:81:in `send'
    /Users/barry/.rvm/gems/ree-1.8.7-2010.02/gems/activesupport-3.1.0/lib/active_support/callbacks.rb:81:in `run_callbacks'
    /Users/barry/.rvm/gems/ree-1.8.7-2010.02/gems/activesupport-3.1.0/lib/active_support/testing/setup_and_teardown.rb:65:in `run'

Finished in 0.005216 seconds.

2 tests, 0 assertions, 0 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications
0% passed

383.44 tests/s, 0.00 assertions/s

Notice how the one failure shows in the results as 1 errors. But the successful assertion does not show, rather 0 assertions.

System info:

~/tmp $ gem list | grep wrong
wrong (0.6.0, 0.5.6, 0.5.4, 0.5.0, 0.4.5)
~/tmp $ gem list | grep test-unit
test-unit (2.4.0, 2.3.2, 2.2.0, 2.1.2, 1.2.3)
~/tmp $ gem list | grep activesupport
activesupport (3.1.0, 3.0.10, 3.0.6, 3.0.5, 3.0.4, 3.0.3, 3.0.1, 3.0.0, 3.0.0.rc, 2.3.8, 2.3.5)

Am I nuts?

thenrio commented 12 years ago

Thank you

The output without wrong should be

2 tests, 2 assertions, 1 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications

You can obtain it with replacing

require 'wrong'

with

require 'test/unit'
require 'wrong/adapters/test_unit'

We will see that require 'wrong' detects and extends loaded test libs (test/unit, minitest, ...) ... or that documentation is improved

bjhess commented 12 years ago

Thanks, Thierry. That did it.

Certainly a simple require 'wrong' would be ideal, but I leave it to you. Thanks again!

alexch commented 12 years ago

Is ActiveSupport::TestCase a subclass of Test::Unit? If so, replace "require 'wrong'" with "require 'wrong/adapters/test_unit'" as it says in the "adapters" section of the readme.

On Thu, Oct 27, 2011 at 11:55 AM, Barry Hess reply@reply.github.com wrote:

I have a test file that is comprised entirely of wrong assertions (wrong_wrong.rb):

   require 'active_support'    require 'wrong'

   class WrongWrongTest < ActiveSupport::TestCase      include Wrong

     def test_if_wrong_is_right        assert{ true }      end

     def test_if_wrong_is_right_about_being_wrong        assert{ false }      end    end

I run the tests:

   ~/tmp $ testrb wrong_wrong.rb    Loaded suite WrongWrongTest    Started    E

   Error:    test_if_wrong_is_right_about_being_wrong(WrongWrongTest):    Wrong::Assert::AssertionFailedError: Expected false        /Users/barry/.rvm/gems/ree-1.8.7-2010.02/gems/wrong-0.6.0/lib/wrong/assert.rb:72:in aver'        /Users/barry/.rvm/gems/ree-1.8.7-2010.02/gems/wrong-0.6.0/lib/wrong/assert.rb:34:inassert'        /Users/barry/tmp/wrong_wrong.rb:12:in test_if_wrong_is_right_about_being_wrong'        /Users/barry/.rvm/gems/ree-1.8.7-2010.02/gems/activesupport-3.1.0/lib/active_support/testing/setup_and_teardown.rb:67:insend'        /Users/barry/.rvm/gems/ree-1.8.7-2010.02/gems/activesupport-3.1.0/lib/active_support/testing/setup_and_teardown.rb:67:in run'        /Users/barry/.rvm/gems/ree-1.8.7-2010.02/gems/activesupport-3.1.0/lib/active_support/callbacks.rb:390:in_run_setup_callbacks'        /Users/barry/.rvm/gems/ree-1.8.7-2010.02/gems/activesupport-3.1.0/lib/active_support/callbacks.rb:81:in send'        /Users/barry/.rvm/gems/ree-1.8.7-2010.02/gems/activesupport-3.1.0/lib/active_support/callbacks.rb:81:inrun_callbacks'        /Users/barry/.rvm/gems/ree-1.8.7-2010.02/gems/activesupport-3.1.0/lib/active_support/testing/setup_and_teardown.rb:65:in `run'

   Finished in 0.005216 seconds.

   2 tests, 0 assertions, 0 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications    0% passed

   383.44 tests/s, 0.00 assertions/s

Notice how the one failure shows in the results as 1 errors. But the successful assertion does not show, rather 0 assertions.

System info:

   ~/tmp $ gem list | grep wrong    wrong (0.6.0, 0.5.6, 0.5.4, 0.5.0, 0.4.5)    ~/tmp $ gem list | grep test-unit    test-unit (2.4.0, 2.3.2, 2.2.0, 2.1.2, 1.2.3)    ~/tmp $ gem list | grep activesupport    activesupport (3.1.0, 3.0.10, 3.0.6, 3.0.5, 3.0.4, 3.0.3, 3.0.1, 3.0.0, 3.0.0.rc, 2.3.8, 2.3.5)

Am I nuts?

Reply to this email directly or view it on GitHub: https://github.com/sconover/wrong/issues/14

Alex Chaffee - alex@cohuman.com - http://alexch.github.com Stalk me: http://friendfeed.com/alexch | http://twitter.com/alexch | http://alexch.tumblr.com

bjhess commented 12 years ago

Right - this is what Thierry shared, correct?

I don't think the README is very clear on the difference between using and not using the adapters. After the fact, it makes sense that wrong is better able to work with the Test::Unit output sugar given direct use of that specific adapter. Loving wrong is all I'm here to do. :)

Thanks!

alexch commented 12 years ago

Steve started Wrong with a less-magic, "make the user require what he wants" attitude, but I'm more a Pit of Success kind of guy... so maybe we should make "require 'wrong'" check all the various test frameworks and if they're loaded, then load the adapters.

http://www.codinghorror.com/blog/2007/08/falling-into-the-pit-of-success.html

On Fri, Oct 28, 2011 at 2:14 PM, Barry Hess reply@reply.github.com wrote:

Right - this is what Thierry shared, correct?

I don't think the README is very clear on the difference between using and not using the adapters. After the fact, it makes sense that wrong is better able to work with the Test::Unit output sugar given direct use of that specific adapter. Loving wrong is all I'm here to do. :)

Thanks!

Reply to this email directly or view it on GitHub: https://github.com/sconover/wrong/issues/14#issuecomment-2561247

Alex Chaffee - alex@cohuman.com - http://alexch.github.com Stalk me: http://friendfeed.com/alexch | http://twitter.com/alexch | http://alexch.tumblr.com

bjhess commented 12 years ago

Ah, I hadn't seen that article before, Alex. Thanks for sharing.

I leave it to you guys - wrong is a great library! I may even fire some pull requests your way after I've used it for a bit longer. (For instance, I'd like to see the ActiveRecord errors object if an assertion fails on an instance of ActiveRecord.)

Feel free to close this issue out. We're just talking now. :)