openclimatefix / pv-pseudo-experiments

Repo containing the training and experimentation code for PV MetNet and Pseudo-Irradiance models
MIT License
2 stars 0 forks source link

Irradiance Model Results #5

Open jacobbieker opened 1 year ago

jacobbieker commented 1 year ago

Here are some overviews of the results from Pseudo-irradiance model training

Detailed Description

The models are trained with batches of 256 (Batch of 8 accumulated 32 times), and 256km areas around random PV sites.

Current Issues

The loss drops dramatically quickly then stays flat, regardless of batch norm, number of examples, channels, etc.

Current (May 2nd) best model is located here

jacobbieker commented 1 year ago

Plots from May 2nd model Validation for batch 1: Inputs (First are NWP, then Sat, HRV, Topo, Sun position): media_images_Input_val_1_88332_f156ddcb678f4181b001 GT: media_images_GT_val_1_88333_910f3703576293cde45f Prediction: media_images_Pred_val_1_88334_49fa69ab098b64ee5b65 Losses on steps: W B Chart 02_05_2023, 09_42_48 W B Chart 02_05_2023, 09_43_00

Notes on this run: The predictions are really odd with the predictions sometimes being blank for future timesteps but other times having values for nearly every pixel. The ground truth doesn't seem to have anything odd about it, and the loss only goes backwards where there are PV systems, so the many zeroe-d out pixels shouldn't matter, unless that means that parts of the network never see PV systems and so don't get updated?

jacobbieker commented 1 year ago

New Idea from ML deep dive chat: Train to predict a single site in the middle of the image, or as a 1D input. Might be simpler and easier to train/debug, and isn't as hard as predicting an image of sites. Could still give the same kinds of inputs, but instead of output image, just give a single tilt and orientation for site in center to get its result. Probably have lower number of pixels to make it actually feasible at all too (i.e. 8x8 or 16x16). Or get predictions for the past points as inputs (i.e. predict for 24 hours ago by giving the model the hour before image, the hour of image, and hour in future image) to help with irradiance then.