assert's failures count as errors

[thegengen]

Hi again.

Using Ruby 1.8.7 and test/unit running this file:

require 'rubygems'
require 'test/unit'
require 'wrong'

class WrongTest < Test::Unit::TestCase
  include Wrong
  def test_something
    assert { false }
  end
end

Gives me this output:

Loaded suite wrong_test
Started
E
Finished in 0.003617 seconds.

  1) Error:
test_something(WrongTest):  
Wrong::Assert::AssertionFailedError: Expected false
    /Users/minciue/.rvm/gems/ree-1.8.7-2011.03/gems/wrong-0.5.0/lib/wrong/assert.rb:68:in `aver'
    /Users/minciue/.rvm/gems/ree-1.8.7-2011.03/gems/wrong-0.5.0/lib/wrong/assert.rb:34:in `assert'
    wrong_test.rb:8:in `test_something'

1 tests, 0 assertions, 0 failures, 1 errors

Thank you.

[thenrio]

And what do you see with a different require ?

require 'rubygems'
require 'test/unit'
require 'wrong/adapters/test_unit'

class WrongTest < Test::Unit::TestCase
  def test_something
    assert { false }
  end
end

Something like

1 tests, 0 assertions, 1 failures, 0 errors

?

~/src/ruby/wrong (master)$ ruby --version
ruby 1.8.7 (2010-12-23 patchlevel 330) [i686-darwin10.6.0]

wrong => 0.5.1

?, Thierry

[thegengen]

Hi.

I was using wrong 0.5.0. The version you're using seems not to have propagated to rubygems.org yet. I cloned your repo and installed 0.5.1 by hand, and the issue (as I reported it) still exists.

After changing the require, I do indeed get a failure, just like in your output.

Oh, and my ruby version:

test [ree-1.8.7] :) ruby --version
ruby 1.8.7 (2011-02-18 patchlevel 334) [i686-darwin10.7.1], MBARI 0x6770, Ruby Enterprise Edition 2011.03

I'll investigate this further myself.

Thanks, Eugen.

[thenrio]

the wrong assert {} raises a default Wrong::Error::AssertionFailedError there is no chance that a test lib understand it as 'failure += 1'

You have to pick the correct adapter for a test lib

Have a look in README : #Adapters WDYT?

[thegengen]

I see, that makes sense.

As for my thoughts, I guess it depends how far you want to go. In theory, require 'wrong' could test for things like

Object.const_defined? :RSpec

etc. and figure out which adapter(s) to load on the fly. You could obtain the current functionality with something like

require 'wrong/base'

If not, I think the documentation should be changed a bit. It should be clear from the #Usage section in the README that you'll probably want to require one of these adapters. I imagine most people just want to hook it into whatever they're using. Not loading any adapters and using wrong solo (as a kind of micro-framework if I understand it correctly) seems to me like only one of the various usage scenarios.

Let me know what you think, maybe I can help you guys out with a patch.

[alexch]

The README already says:

---

Adapters

Adapters for various test frameworks sit under wrong/adapters.

Currently we support

• Test::Unit - require 'wrong/adapters/test_unit'
• Minitest - require 'wrong/adapters/minitest'
• RSpec - require 'wrong/adapters/rspec' (now supports both 1.3 and 2.0)

To use these, put the appropriate require in your helper, after requiring your test framework; it should extend the framework enough that you can use assert { } in your test cases without extra fussing around.

---

Perhaps we should move that section higher up.

[alexch]

BTW I'm OK with autoloading the adapter based on const_defined like minciue suggests.

[alexch]

README updated