Yelp / Testify

A more pythonic testing framework.
Other
306 stars 67 forks source link

_testify_bucket should validate bucket_count #209

Closed mrtyler closed 8 years ago

mrtyler commented 10 years ago

While working on #206, I discovered this.

[tyler@toaster:~/Testify]$ PYTHONPATH=~/Testify ~/Testify/bin/testify -v tmp2.py --bucket=1
Traceback (most recent call last):
  File "/Users/tyler/Testify/bin/testify", line 25, in <module>
    test_program.TestProgram()
  File "/Users/tyler/Testify/testify/test_program.py", line 222, in __init__
    self.run()
  File "/Users/tyler/Testify/testify/test_program.py", line 299, in run
    result = runner.run()
  File "/Users/tyler/Testify/testify/test_runner.py", line 133, in run
    for test_case in self.discover():
  File "/Users/tyler/Testify/testify/test_runner.py", line 108, in discover
    discovered_tests = list(discover_inner())
  File "/Users/tyler/Testify/testify/test_runner.py", line 93, in discover_inner
    or (override_bucket is None and test_case_class._testify_bucket(self.bucket_count, self.bucket_salt) == self.bucket)
  File "/Users/tyler/Testify/testify/test_case.py", line 80, in _testify_bucket
    return hash(MetaTestCase._cmp_str(cls)) % bucket_count
TypeError: unsupported operand type(s) for %: 'int' and 'NoneType'

The offending code:

    def _testify_bucket(cls, bucket_count, bucket_salt=None):
        """Bucket a TestCase using a relatively consistant hash - for dividing tests across runners."""
        if bucket_salt:
            return hash(MetaTestCase._cmp_str(cls) + bucket_salt) % bucket_count
        else:
            return hash(MetaTestCase._cmp_str(cls)) % bucket_count

Called during discovery.

Not sure if it's related but another oddity that I'm not going to chase further is what happened when I provided a bucket-count:

[tyler@toaster:~/Testify]$ PYTHONPATH=~/Testify ~/Testify/bin/testify -v tmp2.py --bucket=1 --bucket-count=1

No tests were discovered (tests must subclass TestCase and test methods must begin with 'test').
ERROR.  0 tests / 0 cases: 0 passed, 0 failed.  (Total test time 0.00s)
asottile commented 10 years ago

After changing buckets this still is a bug but manifests in a different way:

$ testify test --bucket=1
Traceback (most recent call last):
  File "/home/anthony/workspace/Testify/venv/bin/testify", line 10, in <module>
    execfile(__file__)
  File "/home/anthony/workspace/Testify/bin/testify", line 25, in <module>
    test_program.main()
  File "/home/anthony/workspace/Testify/testify/test_program.py", line 339, in main
    sys.exit(not TestProgram().run())
  File "/home/anthony/workspace/Testify/testify/test_program.py", line 303, in run
    return runner.run()
  File "/home/anthony/workspace/Testify/testify/test_runner.py", line 164, in run
    for test_case in self.discover():
  File "/home/anthony/workspace/Testify/testify/test_runner.py", line 137, in discover
    discovered_tests = discover_tests_by_buckets()
  File "/home/anthony/workspace/Testify/testify/test_runner.py", line 118, in discover_tests_by_buckets
    range(self.bucket_count) + list(reversed(range(self.bucket_count)))
TypeError: range() integer end argument expected, got NoneType.
asottile commented 8 years ago

Still reproduces

asottile commented 8 years ago

Via #328