Yelp / Testify

A more pythonic testing framework.
Other
304 stars 67 forks source link

--failure-limit is n *classes* instead of n *tests* when using --rerun-test-file... #319

Closed asottile closed 8 years ago

asottile commented 8 years ago
$ cat tests.txt 
test2 Test.test1
test2 Test.test2
#$ cat test2.py
import testify as T

class Test(T.TestCase):
    def test1(self):
        assert False
    def test2(self):
        assert False

Expected when running alone:

$ testify --failure-limit 1 test2.py
fail: test2 Test.test1
Traceback (most recent call last):
  File "./test2.py", line 5, in test1
    assert False
AssertionError

F
FAILED.  1 test / 1 case: 0 passed, 1 failed.  (Total test time 0.00s)

Buggy when run with --rerun:

$ testify --rerun tests.txt --failure-limit 1
fail: test2 Test.test1
Traceback (most recent call last):
  File "./test2.py", line 5, in test1
    assert False
AssertionError

Ffail: test2 Test.test2
Traceback (most recent call last):
  File "./test2.py", line 7, in test2
    assert False
AssertionError

F
FAILED.  2 tests / 1 case: 0 passed, 2 failed.  (Total test time 0.00s)