search

openclimatefix / satflow

Satellite Optical Flow with machine learning models
https://satflow.readthedocs.io/en/stable/
MIT License
61 stars 10 forks source link

Add Joint Model #101

Open jacobbieker opened 3 years ago

jacobbieker commented 3 years ago

Detailed Description

We want a model that can predict both satellite imagery and pv yield.

Context

While we ultimately just care about the PV output of the model, predicting where the clouds are moving in the satellite imagery should be a good auxiliary task that might help the model learn what is important vs not.

Possible Implementation

Add another version of the Perceiver that takes two queries, one for the satellite imagery and one for PV output.

JackKelly commented 3 years ago

Sounds good!

Let's use this new issue to discuss the joint model. I'll close some older issues where we've discussed related ideas:

As I mumbled about on the meeting this afternoon, I think one of the first steps towards the 'joint model' is to get the position encoding right for the recent history of the PV power data. For each measurement of historical PV power, we definitely want to encode the position in time and space, probably using the same encoding as for the satellite data (so the model can associate historical PV data with historical clouds: it can see "ooh, look, when that cloud passed over that PV system half an hour ago, it reduced power output by 50%").

We may also want the PV "position" encoding to encode an embedding of the ID of the PV system ("ooh, I remember that PV system was unusually dirty, so I'll compensate for that")

(Maybe that "PV-position-encoding" code should go into nowcasting_utils, so predict_pv_yield can use it?)

jacobbieker commented 3 years ago

Yeah, I think breaking out the PV position encoding, and maybe the position encoding in general should probably go into nowcasting_utils I've opened one here https://github.com/openclimatefix/nowcasting_utils/issues/30