CleanCut / green

Green is a clean, colorful, fast python test runner.
MIT License
791 stars 75 forks source link

green does not report broken setUpClass #121

Closed LeoHuckvale closed 8 years ago

LeoHuckvale commented 8 years ago

Say I have a simple test module like:

# test_bad.py
import logging
from unittest import TestCase

log = logging.getLogger('TEST')

class TestFoo(TestCase):
    @classmethod
    def setUpClass(cls):
        log.info('TestFoo setUpClass')
        assert False

    def test_foo(self):
        self.fail('This should never run')

When I run this under the unittest module, I get:

$ python -m unittest test_bad
E
======================================================================
ERROR: setUpClass (test_bad.TestFoo)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_bad.py", line 11, in setUpClass
    assert False
AssertionError

----------------------------------------------------------------------
Ran 0 tests in 0.000s

When I run it under green (2.4.0):

$ green test_bad

Ran 0 tests in 0.107s

No Tests Found

Green reports that it ran 0 tests, consistent with unittest, but I have no way of knowing that my test suite is broken (unless I have logging configured under setUpClass).

With verbosity and debug logs on, I can see that green has successfully found my tests, but it doesn't say that they have errors:

$ green test_bad -vvvd
2016-05-06 14:17:21     DEBUG Attempting to load target 'test_bad' with file_pattern 'test*.py'
2016-05-06 14:17:21     DEBUG Load method: DOTTED OBJECT - test_bad
2016-05-06 14:17:21     DEBUG Found 1 test for target 'test_bad'
Green 2.4.0, Coverage 4.0.2, Python 2.7.11

2016-05-06 14:17:21     DEBUG Attempting to load target 'test_bad' with file_pattern 'test*.py'
2016-05-06 14:17:21     DEBUG Load method: DOTTED OBJECT - test_bad
2016-05-06 14:17:21     DEBUG Found 1 test for target 'test_bad'
2016-05-06 14:17:21      INFO TestFoo setUpClass
Ran 0 tests in 0.109s

No Tests Found
LeoHuckvale commented 8 years ago

Awesome, thanks!