Read last year competitions' papers for cool ideas

[bruce-willis]

Weather4cast

(overview) High-resolution multi-channel weather forecasting – First insights on transfer learning from the Weather4cast Competitions 2021 IEEE

In contrast, however, the winning submission did not employ a U-Net but a recurrent convolutional network with residual units.

we simply collapse the time dimension T into the channels dimension C, thereby enlarging the depth of the channels: [T, C, W, H] → [T × C, W, H].

All teams trained with all regions at once and report this as a major factor of improvement.

The three top-ranked submissions all used model ensembles. The ensemble consistently outperformed the prediction from the single model with the best validation score

For each place usually 2 links from 2 stages, the second one is more comprehensive usually

[bruce-willis]

1st: Spatiotemporal Weather Data Predictions with Shortcut Recurrent-Convolutional Networks: A Solution for the Weather4cast challenge arxiv, Improvements to short-term weather prediction with recurrent-convolutional networks arxiv

The model presented here is a neural network combining recurrent-convolutional layers and shortcut connections in an encoder-forecaster architecture.

Convolutional GRU

Data augmentation by random rotation in 90° increments as well as random top-down and left-right mirroring was used to further increase the effective number of training samples.

The validation set had an average crr intensity of 7.26 × 10−4 on the [0, 1] normalized scale, while for the training set it was considerably lower, 6.01 × 10−4 . The equivalent discrepancies for the other variables were at least an order of magnitude smaller. The problem of different distributions in the training and validation datasets was approached by building a conditional model containing two copies of the same model. The results from one of these copies would be used if the maximum rain rate in the input time series was less than a threshold value of 0.026 on the [0,1] normalized scale; otherwise the other copy would be used. The threshold is was chosen such that it divides the training dataset approximately evenly over the two models. The conditional approach helps adjust for the representativeness problem because, as the validation set contains more raining cases than the training set, the model used for the higher rain rates would be used more often to compensate for this.

While the original worked used the widely adopted Adam optimizer, in this work AdaBelief [10], a modification of the algorithm used in Adam [11], was adopted.

First, it can be seen that simply retraining the model with the new data and the AdaBelief optimizer improved the results considerably (up to 4.8% for cma)

Third, the ensembling further improves the scores for all variables

[bruce-willis]

2nd: Utilizing unet for the future weather prediction: Weather4cast 2021 ceur, UNet based future weather prediction on Weather4cast 2021 stage 2 IEEE

We trained neural network model having UNet based architecture. Each convolution block is densely connected with subsequent layers like a DenseNet

In this study, recurrent model such as LSTM (long short term memory network) was not utilized.

So, time dimension is simply merged into feature channel dimension.

All training weather images from different re- gions were merged and used together to train general model that can be used to apply on all regions.

We trained multiple UNets and combined predictions from individual models.

[bruce-willis]

3rd: A Variational U-Net for Weather Forecasting arxiv, Enhanced Variational U-Net for Weather Forecasting IEEE

[bruce-willis]

5th: Spatiotemporal Swin-Transformer Network for Short Time Weather Forecasting ceur, Spatiotemporal Vision Transformer for Short Time Weather Forecasting IEEE

author's papers