pySTEPS / pysteps

Python framework for short-term ensemble prediction systems.
https://pysteps.github.io/
BSD 3-Clause "New" or "Revised" License
473 stars 168 forks source link

Modify probability matching code to allow ignoring a part of the domain #428

Closed ladc closed 3 months ago

ladc commented 3 months ago

The resampling now allows the presence of nans. For pixel locations with a nan, both arrays are ignored. The probability matching has an extra argument ignore_indices which are left untouched.

This functionality is added so that the values outside the radar mask, where there is only NWP data, don't affect the distribution of rainfall inside the mask. In the case of high intensity in NWP but not in the radar for example, you'd otherwise get a huge jump in intensity due to the extremes affecting the radar nowcast area.

Result is in the top right and bottom left: control_ens_mean_obs

RubenImhoff commented 3 months ago

Fantastic, @ladc! I will have a look at it after the weekend. :)

ladc commented 3 months ago

Thanks! There's still somewhat of a jump, but not sure if we can entirely avoid it.

codecov[bot] commented 3 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 84.00%. Comparing base (b4b41a5) to head (df2ce0e). Report is 1 commits behind head on master.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #428 +/- ## ========================================== + Coverage 83.85% 84.00% +0.14% ========================================== Files 160 160 Lines 12780 12882 +102 ========================================== + Hits 10717 10821 +104 + Misses 2063 2061 -2 ``` | [Flag](https://app.codecov.io/gh/pySTEPS/pysteps/pull/428/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pySTEPS) | Coverage Δ | | |---|---|---| | [unit_tests](https://app.codecov.io/gh/pySTEPS/pysteps/pull/428/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pySTEPS) | `84.00% <100.00%> (+0.14%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pySTEPS#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

dnerini commented 3 months ago

Quick thought: would it perhaps help to simply "fade in" in few time steps after t0 from zeros to the NWP values outside the radar mask? This way, you would ensure a smooth transition

Edit: on a second thought and more careful read, you're approach achieves the same in a more elegant way, way to go!

ladc commented 3 months ago

Here's just a final animation with and without the smoothing between nwc and nwp (mask boundary fix by Simon) row 1: "control" (other branch, not yet finished) - member - mean (without smoothing) row 2: observation - member with smoothing - mean with smoothing

control_ens_mean_obs_enssmooth_meansmooth