Yelp / Testify

A more pythonic testing framework.
Other
306 stars 67 forks source link

run tests in longest-first order #278

Closed bukzor closed 8 years ago

bukzor commented 10 years ago

@osarood has demonstrated a 20% real-world time savings in his proof-of-concept branch here: https://github.com/osarood/Testify/commits/osarood_new_batching

The task now is to bring that branch up to production quality (ie add assertions of the behavior).

A 20% savings for all of our yelp-main developers is very significant; this should be considered high priority.

osarood commented 10 years ago

Its 20% using the same number of slaves. The reason for fixing this issue was to enable testify to scale to more than 7 slaves. This branch removes the problem of choking testify server and enables it to scale to more than 7 slaves.

I was able to do runs where testify part of a BB run took less than 13 mins compared to 25-30 mins (current time). So, in essence, we can reduce the execution time by 50% using this branch.

Osman P.S: There is this buildbot flakiness that causes slaves to not-wake-up which creates problems as you increase slaves.

On Tue, Aug 5, 2014 at 10:09 AM, Buck Golemon notifications@github.com wrote:

@osarood https://github.com/osarood has demonstrated a 20% real-world time savings in his proof-of-concept branch here: https://github.com/osarood/Testify/commits/osarood_new_batching

The task now is to bring that branch up to production quality (ie add assertions of the behavior).

A 20% savings for all of our yelp-main developers is very significant; this should be considered high priority.

— Reply to this email directly or view it on GitHub https://github.com/Yelp/Testify/issues/278.

bukzor commented 10 years ago

Internal ID: BUILDBOT-255

asottile commented 8 years ago

wontfix, the client/server mode is no more!