search

Yelp / Testify

A more pythonic testing framework.
Other
306 stars 67 forks source link

More fair round-robin test bucketing. #216

Closed asottile closed 10 years ago

asottile commented 10 years ago

Resolves https://github.com/Yelp/Testify/issues/215

There were 0 tests of bucketing prior to this so I added some for this change.

asottile commented 10 years ago

Updated with some documentation of the algorithm in the test.

asottile commented 10 years ago

Oh, I thought I'd include some anecdotal stats:

master:

(herpderp)asottile@dev5-devc:~/Testify (master) $ bin/testify test --list-tests --bucket-count=5 --bucket=0 | wc -l
87
(herpderp)asottile@dev5-devc:~/Testify (master) $ bin/testify test --list-tests --bucket-count=5 --bucket=1 | wc -l
16
(herpderp)asottile@dev5-devc:~/Testify (master) $ bin/testify test --list-tests --bucket-count=5 --bucket=2 | wc -l
35
(herpderp)asottile@dev5-devc:~/Testify (master) $ bin/testify test --list-tests --bucket-count=5 --bucket=3 | wc -l
51
(herpderp)asottile@dev5-devc:~/Testify (master) $ bin/testify test --list-tests --bucket-count=5 --bucket=4 | wc -l
95

My branch:

(herpderp)asottile@dev5-devc:~/Testify (more_fair_buckets) $ bin/testify test --list-tests --bucket-count=5 --bucket=0 | wc -l
61
(herpderp)asottile@dev5-devc:~/Testify (more_fair_buckets) $ bin/testify test --list-tests --bucket-count=5 --bucket=1 | wc -l
60
(herpderp)asottile@dev5-devc:~/Testify (more_fair_buckets) $ bin/testify test --list-tests --bucket-count=5 --bucket=2 | wc -l
62
(herpderp)asottile@dev5-devc:~/Testify (more_fair_buckets) $ bin/testify test --list-tests --bucket-count=5 --bucket=3 | wc -l
62
(herpderp)asottile@dev5-devc:~/Testify (more_fair_buckets) $ bin/testify test --list-tests --bucket-count=5 --bucket=4 | wc -l
63
mrtyler commented 10 years ago

Nice $tats!

ShipIt!

(I'll give you a minute to fix that trailing newline before I merge to master ;))