Closed ladc closed 4 months ago
Dear Michiel and Lesley
Many thanks for starting this important discussion. (let's ping @pulkkins @RubenImhoff @loforest directly here too)
For radar only nowcasts, the problem is somewhat simpler. Some routines will have no problems in dealing with all zeros: e.g. the LK optical flow will simply return a field of zero displacement. Others, the steps nowcasts for example, will fail (as noted in #282).
I guess that in the past we have mostly dealt with this before calling the main nowcasting routine: if no precipitation is observed by radar in the recent past, do not run the nowcast as it will be all zeros anyway. I acknowledge that it would be better to deal with such cases within the nowcasting method itself (if input is all zeros, return a forecast with all zeros), or at least raise a clear error message. Clearly, this starts by adding tests cases for nowcasting with input without precipitation.
For blending this gets more complicated, as you nicely summarized above. Maybe we should ask directly Alan? Surely he had to find a solution for Australia ...
Hi Michiel, Lesley and Daniele,
Very relevant points that we come across often in operational systems..
I think that the proposed options by you and Daniele (also in his PR) are worth the try. Regarding having no AR parameters because of no rain in the rainfall estimates for the nowcasting part (in STEPS nowcasting or blending), would it be an option to have a "base set" of parameters in case the model returns nothing (when there is no rain)? In any case, let's see what Alan and colleagues have done at BoM. Perhaps they have come up with a similarish solution.
I like your idea for when the QPE of the observation and the NWP forecasts are zero. This may be worth the try, although I find it hard to judge how much implementation time this will take - i.e., is it an easy try out or a lot of work to test the idea?
@ladc and @Michiel, any updates about this issue? Or is this something we should try to pick up together soon?
Hello Ruben,
Not yet, indeed. This could be one of the issues to tackle during the pysteps workshop :-) I'll check this with Alan, I had not yet done so.
Hello everyone,
I summarize Alan's reply below
He also noted that STEPS1 kept running mean (30d) for the parameters; transition from observed to climatological parameters over 6h (this is implemented in our pysteps blending method, but Ruben found that 3d gave a better performance).
I think we can now consider this issue solved, thanks everyone!
Hi @ladc great to hear that you managed to implement a solution! Are you planning to merge those contributions back to pysteps soon? Till then, I'd rather keep this issue open
Yes, you're right, closing the issue was a bit premature :-) I think these changes can be merged back after review, but it does change some default behaviour of the blending, so a new minor version number should be assigned.
Dear developers,
As mentioned in #282 and #295, the STEPS nowcasting component of pysteps (with/without blending with NWP) cannot really handle the case where there is no rain in the radar image / QPE. The problems are not only technical but also scientific: how to determine optical flow and AR/scaling parameters from a field of zeroes? And subsequently, how to extrapolate/blend? We see 2 possible problem scenarios:
QPE is zero and NWP is zero: This is a bigger problem since there's simply no rain to calculate any optical flow or noise parameters. One could immediately use the NWP instead of blending, but that does not yet give us a way to generate variability in the ensemble forecast. A possible solution is to use climatological values (or values of the past X days) to determine the noise parameters. That way, you can generate some noise in the ensemble members (to generate variability when it actually starts to rain in the NWP forecast).
What are your thoughts on this?
Cheers,
Michiel and Lesley