Hyperparameter tuning PVNet UK - Phase I

[peterdudfield]

Hyper parameter tune PVnet for the UK.

To discuss which parameters should be adjusted

[Sukh-P]

Kicked off some hyperparameter tuning, looking into:

• Changing parameters linked to the NWP encoders like altering the number of conv3d channels, number of out features If there is time:
• Changing the weight decay/regularisation value
• Changing the number of embedding dimensions for the GSP encoding

[Sukh-P]

Completed a first round of hyperparameter tuning experiments with some modest improvements (relative 3% improvement in normalised MAE compared to the current PVNet configuration) were seen on the small experiments I ran when using larger models/encoders for the NWP parts, definitely not conclusive due to caveats mentioned in the doc but perhaps enough to revisit this in the future and run a test on a model trained for longer.

See this doc for a full writeup of what the experiments were and results.