Yelp / Testify

A more pythonic testing framework.
Other
306 stars 67 forks source link

Refactor bucketing to be more fair #215

Closed asottile closed 10 years ago

asottile commented 10 years ago

Currently bucketing is done on the hash of the name. I think it would be more fair if something along these lines happened:

(This is of course pseudocode but you get the idea)


buckets = collections.defaultdict(list)

# Sort by number of tests, then test name or hash of test name (like the current implementation does) (so it is deterministic)
test_cases = sorted(
    test_cases,
    key=lambda test_case: (test_case.number_of_tests, test_case.__name__),
)

# Round robin insert into buckets
test_cases_iter = iter(test_cases)
try:
    while True:
        for bucket_number in xrange(bucket_count):
            buckets[bucket_number].append(test_cases_iter.next())
except StopIteration:
    # Cool we're done
    pass
asottile commented 10 years ago

as @bukzor notes:


# ...

for bucket, test_case in itertools.izip(itertools.cycle(range(buckets)), test_cases):
    buckets[bucket].append(test_case)

is more reasonable