JetBrains / teamcity-messages

Python Unit Test Reporting to TeamCity
https://pypi.python.org/pypi/teamcity-messages
Apache License 2.0
137 stars 81 forks source link

unittest breaks "fastfail" (and probably other) options #135

Closed throwable-one closed 7 years ago

throwable-one commented 7 years ago
import unittest
from unittest import TestCase

from teamcity.unittestpy import TeamcityTestRunner

class FooTest(TestCase):
    def test_1_test(self):
        pass

    def test_2_test(self):
        self.fail()

    def test_3_test(self):
        pass

if __name__ == '__main__':
    runner = TeamcityTestRunner()
    unittest.main(testRunner=runner)

Launched from console

"c:\Program Files\Python35\python.exe" -m unittest test_test.py -f -v
test_1_test (test_test.FooTest) ... ok
test_2_test (test_test.FooTest) ... FAIL

======================================================================
FAIL: test_2_test (test_test.FooTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\temp\breakunittest\test_test.py", line 12, in test_2_test
    self.fail()
AssertionError: None

----------------------------------------------------------------------
Ran 2 tests in 0.001s

FAILED (failures=1)

It says 2 tests (third one skipped)

Launched directly

C:\temp\breakunittest>"c:\Program Files\Python35\python.exe" test_test.py -f -v
##teamcity[testCount timestamp='2017-04-18T18:33:29.619' count='3']
##teamcity[testStarted timestamp='2017-04-18T18:33:29.619' captureStandardOutput='true' flowId='__main__.FooTest.test_1_test' name='__main__.FooTest.test_1_test']
##teamcity[testFinished timestamp='2017-04-18T18:33:29.619' duration='0' flowId='__main__.FooTest.test_1_test' name='__main__.FooTest.test_1_test']
##teamcity[testStarted timestamp='2017-04-18T18:33:29.619' captureStandardOutput='true' flowId='__main__.FooTest.test_2_test' name='__main__.FooTest.test_2_test']
##teamcity[testFailed timestamp='2017-04-18T18:33:29.621' details='Traceback (most recent call last):|n  File "c:\Program Files\Python35\lib\unittest\case.py", line 59, in testPartExecutor|n    yield|n  File "c:\Prog
ram Files\Python35\lib\unittest\case.py", line 601, in run|n    testMethod()|n  File "test_test.py", line 12, in test_2_test|n    self.fail()|n  File "c:\Program Files\Python35\lib\unittest\case.py", line 666, in fai
l|n    raise self.failureException(msg)|nAssertionError: None|n' flowId='__main__.FooTest.test_2_test' message='Failure' name='__main__.FooTest.test_2_test']
##teamcity[testFinished timestamp='2017-04-18T18:33:29.621' duration='1' flowId='__main__.FooTest.test_2_test' name='__main__.FooTest.test_2_test']
##teamcity[testStarted timestamp='2017-04-18T18:33:29.621' captureStandardOutput='true' flowId='__main__.FooTest.test_3_test' name='__main__.FooTest.test_3_test']
##teamcity[testFinished timestamp='2017-04-18T18:33:29.621' duration='0' flowId='__main__.FooTest.test_3_test' name='__main__.FooTest.test_3_test']

Ran 3 tests in 0.003s

FAILED (failures=1)

Ran 3 tests. Third one is also launched (##teamcity[testStarted timestamp='2017-04-18T18:33:29.621' captureStandardOutput='true' flowId='__main__.FooTest.test_3_test' name='__main__.FooTest.test_3_test'])

shalupov commented 7 years ago

for Python 3.2 (where all additional options are supported) you need to call unittest.main differently:

    main(argv=args, module=None, buffer=True, testRunner=unittestpy.TeamcityTestRunner)

this way all custom options like failfast will be passed to TextTestRunner which will handle it

throwable-one commented 7 years ago

@shalupov thats exactly what I am doing.

shalupov commented 7 years ago

@throwable-one you're making an instance of TeamcityTestRunner by youself. If you pass type instead of instance unittest will create it itself and pass all relevant arguments (i.e. buffer and failfast)

throwable-one commented 7 years ago

@shalupov Got it, thank you!

throwable-one commented 7 years ago

btw, should not you fix your example code then: https://github.com/JetBrains/teamcity-messages ?