Plug "power-from-imagery Transformer Encoder"'s predictions into a second Transformer Encoder which looks across time.

Pre-train the "satellite image to PV power" model with 5-minutely steps.
Then use 30-minutely steps (at the top and bottom of the hour) so it's easy to figure out which GSP timestep we're using
Try with loops first, as it's easier to implement. Although it maybe be faster to train if we can do the forward pass without loops.
Separate out the "satellite image to PV power" model from the "time transformer" model as separate nn.Modules.
First, try just separating out the models, and loading the weights, and running the thing without the "time transformer".
When the "time Transformer" is completely untrained, maybe lock the "satellite image to PV power" transformer's weights.

openclimatefix / power_perceiver