openclimatefix / power_perceiver

Machine learning experiments using the Perceiver IO model to forecast the electricity system (starting with solar)
MIT License
7 stars 1 forks source link

Implement `PVSystemDataset` #22

Closed JackKelly closed 2 years ago

JackKelly commented 2 years ago

Detailed Description

Some additional notes:

Context

Will be used for #20 (and others)

Implementation

This can't be done in the DataLoader class, because those classes only know about each individual data source. And we need to process the PV data and HRV satellite data together. Some options:

JackKelly commented 2 years ago

OK, it turns out that some examples end up having zero PV systems.

NEXT STEP: Explore using a variable-sized batch (and dropping examples without any PV systems)