Implement first ML model (based on Perceiver IO). It just handles one timestep at a time. It takes an HRV satellite image and predicts PV power for all the PV systems near the middle of the image.

[JackKelly]

• [x] #36
• [x] Build very simple Perceiver model (not Perceiver IO). All code in one file. Inputs: HRV satellite. Each query element: PV location. Output: PV yield for each PV system in the query. Include a variable number of PV systems in the query. Pad the query with a fixed size learnable array (those query elements will be ignored in the output).
• [x] Run model locally and on donatello
• [x] Include embedding of PV system ID in query.
• [x] Split train and validation data
• [x] #37
• [ ] Send plots to wandb:
  • [x] Timeseries plots
  • [x] tSNE of PV system ID embedding
  • [ ] Satellite plots
  • [ ] Attention matrix
• [ ] Experiments:
  • [x] #38
  • [x] Include solar features in query
  • [x] Plot timeseries for multiple batches
  • [x] #42
  • [x] #41
  • [x] #40
  • [x] #44
  • [x] #45
  • [x] In the tSNE plot, use all the PV systems that we're using
  • [ ] #43
  • [x] #47 (This is pretty important. If Perceiver IO doesn't perform as well as the "full multihead attention" then I'll need to think how to incorporate "full multihead attention" into the "MetNet x Hierarchical Perceiver" experiments)
  • [x] Convert to a Python script

[JackKelly]

OK, good news, I'm pretty sure this super-simple model is learning something useful.

Here's a (cherrypicked!) example (from the training set):

And it is producing different predictions for each PV system within an example:

And the PV system embedding is changing for each PV system:

I'll update the plan above

[JackKelly]

This is done. And the "transformer encoder" actually works really well :slightly_smiling_face: