search

materialsproject / atomate2

atomate2 is a library of computational materials science workflows
https://materialsproject.github.io/atomate2/
Other
167 stars 96 forks source link

Use `pytest-split` to parallelize across 3 runners and speedup CI #985

Closed esoteric-ephemera closed 1 month ago

esoteric-ephemera commented 2 months ago

Summary

Splits atomate2 tests into batches using pytest-split (see related pymatgen change a while ago). Test times were generated from a CI run and averaging over python versions. This PR is synced up with #940 per the discussion there. Copying @janosh

Current timing info

The forcefield Gruneisen workflow previously took a very long time, ~845 sec in CI. Making some small tweaks to its BasePhononMaker has reduced the time significantly to about 130 sec. Now the tests run at 310 sec/split for 3 splits

Previous timing info

The best load balancing is probably 2-3 splits, since one workflow (forcefield Gruneisen) takes a very large amount of time

Some splits vs. timing notes:

codecov[bot] commented 2 months ago

Codecov Report

Attention: Patch coverage is 67.09402% with 231 lines in your changes missing coverage. Please review.

Project coverage is 4.73%. Comparing base (8d57884) to head (af1523e). Report is 25 commits behind head on main.

Files with missing lines Patch % Lines
src/atomate2/forcefields/jobs.py 0.00% 54 Missing :warning:
src/atomate2/ase/md.py 76.92% 26 Missing and 16 partials :warning:
src/atomate2/forcefields/md.py 0.00% 32 Missing :warning:
src/atomate2/forcefields/schemas.py 0.00% 26 Missing :warning:
src/atomate2/forcefields/flows/eos.py 0.00% 20 Missing :warning:
src/atomate2/ase/utils.py 87.74% 10 Missing and 9 partials :warning:
src/atomate2/ase/jobs.py 83.33% 9 Missing and 3 partials :warning:
src/atomate2/ase/schemas.py 93.10% 6 Missing and 4 partials :warning:
src/atomate2/forcefields/__init__.py 0.00% 5 Missing :warning:
src/atomate2/forcefields/flows/relax.py 0.00% 3 Missing :warning:
... and 5 more
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #985 +/- ## ========================================== - Coverage 75.70% 4.73% -70.98% ========================================== Files 147 160 +13 Lines 10925 11747 +822 Branches 1613 1742 +129 ========================================== - Hits 8271 556 -7715 - Misses 2173 11157 +8984 + Partials 481 34 -447 ``` | [Files with missing lines](https://app.codecov.io/gh/materialsproject/atomate2/pull/985?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=materialsproject) | Coverage Δ | | |---|---|---| | [src/atomate2/forcefields/flows/elastic.py](https://app.codecov.io/gh/materialsproject/atomate2/pull/985?src=pr&el=tree&filepath=src%2Fatomate2%2Fforcefields%2Fflows%2Felastic.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=materialsproject#diff-c3JjL2F0b21hdGUyL2ZvcmNlZmllbGRzL2Zsb3dzL2VsYXN0aWMucHk=) | `0.00% <0.00%> (-81.82%)` | :arrow_down: | | [src/atomate2/forcefields/flows/gruneisen.py](https://app.codecov.io/gh/materialsproject/atomate2/pull/985?src=pr&el=tree&filepath=src%2Fatomate2%2Fforcefields%2Fflows%2Fgruneisen.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=materialsproject#diff-c3JjL2F0b21hdGUyL2ZvcmNlZmllbGRzL2Zsb3dzL2dydW5laXNlbi5weQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [src/atomate2/forcefields/flows/phonons.py](https://app.codecov.io/gh/materialsproject/atomate2/pull/985?src=pr&el=tree&filepath=src%2Fatomate2%2Fforcefields%2Fflows%2Fphonons.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=materialsproject#diff-c3JjL2F0b21hdGUyL2ZvcmNlZmllbGRzL2Zsb3dzL3Bob25vbnMucHk=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [src/atomate2/forcefields/flows/qha.py](https://app.codecov.io/gh/materialsproject/atomate2/pull/985?src=pr&el=tree&filepath=src%2Fatomate2%2Fforcefields%2Fflows%2Fqha.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=materialsproject#diff-c3JjL2F0b21hdGUyL2ZvcmNlZmllbGRzL2Zsb3dzL3FoYS5weQ==) | `0.00% <0.00%> (ø)` | | | [src/atomate2/forcefields/flows/relax.py](https://app.codecov.io/gh/materialsproject/atomate2/pull/985?src=pr&el=tree&filepath=src%2Fatomate2%2Fforcefields%2Fflows%2Frelax.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=materialsproject#diff-c3JjL2F0b21hdGUyL2ZvcmNlZmllbGRzL2Zsb3dzL3JlbGF4LnB5) | `0.00% <0.00%> (-63.34%)` | :arrow_down: | | [src/atomate2/forcefields/utils.py](https://app.codecov.io/gh/materialsproject/atomate2/pull/985?src=pr&el=tree&filepath=src%2Fatomate2%2Fforcefields%2Futils.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=materialsproject#diff-c3JjL2F0b21hdGUyL2ZvcmNlZmllbGRzL3V0aWxzLnB5) | `0.00% <0.00%> (-91.63%)` | :arrow_down: | | [src/atomate2/forcefields/\_\_init\_\_.py](https://app.codecov.io/gh/materialsproject/atomate2/pull/985?src=pr&el=tree&filepath=src%2Fatomate2%2Fforcefields%2F__init__.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=materialsproject#diff-c3JjL2F0b21hdGUyL2ZvcmNlZmllbGRzL19faW5pdF9fLnB5) | `0.00% <0.00%> (-100.00%)` | :arrow_down: | | [src/atomate2/ase/schemas.py](https://app.codecov.io/gh/materialsproject/atomate2/pull/985?src=pr&el=tree&filepath=src%2Fatomate2%2Fase%2Fschemas.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=materialsproject#diff-c3JjL2F0b21hdGUyL2FzZS9zY2hlbWFzLnB5) | `93.10% <93.10%> (ø)` | | | [src/atomate2/ase/jobs.py](https://app.codecov.io/gh/materialsproject/atomate2/pull/985?src=pr&el=tree&filepath=src%2Fatomate2%2Fase%2Fjobs.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=materialsproject#diff-c3JjL2F0b21hdGUyL2FzZS9qb2JzLnB5) | `83.33% <83.33%> (ø)` | | | [src/atomate2/ase/utils.py](https://app.codecov.io/gh/materialsproject/atomate2/pull/985?src=pr&el=tree&filepath=src%2Fatomate2%2Fase%2Futils.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=materialsproject#diff-c3JjL2F0b21hdGUyL2FzZS91dGlscy5weQ==) | `87.74% <87.74%> (ø)` | | | ... and [5 more](https://app.codecov.io/gh/materialsproject/atomate2/pull/985?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=materialsproject) | | ... and [120 files with indirect coverage changes](https://app.codecov.io/gh/materialsproject/atomate2/pull/985/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=materialsproject)