boutproject / xBOUT

Collects BOUT++ data from parallelized simulations into xarray.
https://xbout.readthedocs.io/en/latest/
Apache License 2.0
21 stars 10 forks source link

Cache input data for tests #215

Closed johnomotani closed 3 years ago

johnomotani commented 3 years ago

When creating input files/Datasets for tests, save them in a cache and return a copy if the same arguments are given to the data-creation function(s). Helps to speed things up a bit - seems to shave about 20mins of the CI runs.

Includes an update changing the use of Pytest's tmpdir_factory and tmpdir for tmp_path_factory and tmp_path. The latter return a pathlib.Path object instead of Pytests's py.path, which is more convenient and compatible with standard Python path-handling. I think those changes are actually most of the changed lines though - for just the caching updates, see 4e4e1427c1c4ac2aeae1e9cbd99e71ccbefd243c and ac191a3a2e45c7370dd8d7c71dfbfe0e9bddcb87.

Partial fix for #213.

johnomotani commented 3 years ago

PS I suspect this helps test runs using pytest --long more than regular runs (without the --long) - there probably aren't as many duplicated inputs to avoid recreating in the regular runs...

codecov-commenter commented 3 years ago

Codecov Report

Merging #215 (535406f) into master (2478fe6) will decrease coverage by 0.01%. The diff coverage is 50.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #215      +/-   ##
==========================================
- Coverage   75.37%   75.35%   -0.02%     
==========================================
  Files          14       14              
  Lines        2644     2646       +2     
  Branches      622      623       +1     
==========================================
+ Hits         1993     1994       +1     
  Misses        421      421              
- Partials      230      231       +1     
Impacted Files Coverage Δ
xbout/load.py 77.99% <50.00%> (-0.16%) :arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 2478fe6...535406f. Read the comment docs.

ZedThree commented 3 years ago

This looks like it shaves a good chunk of time off of the Github Actions at least!

johnomotani commented 3 years ago

Shout out to gelidum, a nice package which converts many kinds of object to hashable things that can be used as dict keys :+1: Now available on conda-forge!

johnomotani commented 3 years ago

PS I'm going to release a new minor version before I forget to update xbout's dependencies on conda-forge.