search

metoppv / improver

IMPROVER is a library of algorithms for meteorological post-processing.
http://improver.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
101 stars 84 forks source link

Speed up rainforests prediction #1966

Closed btrotta-bom closed 5 months ago

btrotta-bom commented 8 months ago

Add an option to speed up rainforests prediction.

The improved efficiency is achieved by observing that for sets of grid points having very similar values for their features, the features will be binned by LightGBM in the same way for each point, and therefore the predicted value of each point will be the same. Thus we only need to predict once for the whole set of points. This yields a reduction of around 20-40% in processing time for gridded forecasts (depending on file).

Description

Testing:

codecov[bot] commented 8 months ago

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (14723f3) 98.40% compared to head (42c1bb6) 98.40%.

Files Patch % Lines
improver/calibration/rainforest_calibration.py 98.59% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #1966 +/- ## ========================================== - Coverage 98.40% 98.40% -0.01% ========================================== Files 124 124 Lines 12069 12125 +56 ========================================== + Hits 11876 11931 +55 - Misses 193 194 +1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

github-actions[bot] commented 6 months ago

In order to maintain a backlog of relevant PRs, we automatically label them as stale after 60 days of inactivity.

If this PR is still important to you, then please comment on this PR and the stale label will be removed.

Otherwise this PR will be automatically closed in 30 days time.

btrotta-bom commented 6 months ago

This is still current.

benowen-bom commented 5 months ago

Scheduled tests are failing with the latest env. We should pin the treelite package in latest.yml to resolve.

btrotta-bom commented 5 months ago

Scheduled tests are failing with the latest env. We should pin the treelite package in latest.yml to resolve.

Done

btrotta-bom commented 5 months ago

Thanks for reviewing @benowen-bom and @bomRob, all changes are merged now.

benowen-bom commented 5 months ago

I realised after merging that the fix to envs/latest.yml to pin the treelite package wasn't included here. I'll add a PR tomorrow to enact this change.