search

ecmwf / earthkit-regrid

2 stars 0 forks source link

WIP: Add in-memory cache for interpolation matrices #23

Open sandorkertesz opened 5 months ago

sandorkertesz commented 5 months ago

This PR adds an in-memory LRU cache for the interpolation matrices. The cache has a memory limit which is set to 300 MB by default. The memory limit can also be set to 0 (no caching) or None (no upper memory limit).

The memory limit configuration is not yet exposed to the users (hardcoded).

With this cache repeated interpolations between the same input and output grids can be run much faster. The speed-up is around x10 on a MacBook using the "largest" interpolation available at the moment:

  earthkit.regrid.interpolate(data, 
                        {"grid": "O2560"}, 
                        {"grid": [0.1,0.1]})
codecov-commenter commented 5 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.57%. Comparing base (3aa638b) to head (f51d06d).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #23 +/- ## =========================================== + Coverage 91.96% 94.57% +2.61% =========================================== Files 7 7 Lines 249 369 +120 Branches 13 13 =========================================== + Hits 229 349 +120 Misses 18 18 Partials 2 2 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.