search

avocado-framework / avocado

Avocado is a set of tools and libraries to help with automated testing. One can call it a test framework with benefits. Native tests are written in Python and they follow the unittest pattern, but any executable can serve as a test.
https://avocado-framework.github.io/
Other
345 stars 344 forks source link

Evaluate the knapsack approach for distributing tests #3953

Open clebergnu opened 4 years ago

clebergnu commented 4 years ago

GitLab is using a "knapsack" approach, and files with data tests, to evenly distribute tests in multiple jobs. It looks like, if nothing is provided by the project/repo, an empty {} is set by default.

The goal here is to try to understand if Avocado can contribute information or use the same information to allow the same level of parallelization of a test suite.

For instance, we know that the libvirt-tck tests can run different types of tests (each major directory) in parallel, if the environments are isolated (what is expected here). So, if that information is set, somehow, Avocado when running with the N(ext) Runner could have each test type in a different VM on a developer's machine and the same would be communicated to GitLab, which would run various jobs.

References:

clebergnu commented 4 years ago

@beraldoleal this seems to be interesting for the libvirt-tck case. @willianrampazzo if you have some spare cycles, maybe you can take a closer look at this and come up with a report on it?