About the precipitation sampling in "Skillful Precipitation Nowcasting Using Deep Generative Models of Radar"

[J-shel]

Hello,

In the paper "Skillful Precipitation Nowcasting Using Deep Generative Models of Radar" you made use of an importance sampling scheme to increase the frequency with which crops with rain were encountered during training. I wonder if you could share the code for it? Thanks a lot!

[mjwillson]

Hello and thanks for the interest!

I'm afraid it's unlikely we'll open source this code as it's rather tied up in our proprietary data processing pipelines.

That said we do have a fairly detailed description of the method in the Supplementary Materials. In section A.1. Importance Sampling Scheme we explain how the sampling probabilities are computed, and in section F (F.1 in particular) it's explained how the inverse sampling probabilities are used as importance weights when computing estimates of verification metrics.

We also include the importance sampling probabilities that we used as a sample_prob feature in the datasets we released, and these can be used to compute the importance-weighted estimates of verification metrics.

Feel free to email us if any of this is unclear.

[tranthanhsongitrepo]

@mjwillson Hi, I am trying to confirm my understand the sampling scheme.

If I understood the paper correctly, the oversampling scheme is done like so: Patches of size Txwxh are cropped from a radar image of size TxHxW. Each patch will have a different probability x_n calculated by using formula (6) in the paper. We sample a bunch of these patches and store them away to use as our training set.

Let's say I draw a patch from this training set, and its x_n is 0.9, does this mean that there's a 0.1 chance that I have to draw another patch and a 0.9 chance of me using this patch for training?