openclimatefix / uk-pvnet-app

Application for running PVNet in production
MIT License
1 stars 5 forks source link

Refactor #133

Closed dfulu closed 2 days ago

dfulu commented 3 days ago

Pull Request

Description

A medium refactor of the code which keeps all of same the functionality and user facing APIs of the old code

Extra fixes:

  1. We use elevation from the batch to compute night time masks. The elevation values from ocf-data-sampler are 0-1 scaled whilst the elevation values from ocf_datapipes are standardised. A fix for this, including legacy flag was added
  2. In the pvnet pipeline in ocf_datapipes with the production flag switched on, when satellite data is missing we allow linear interpolation to fill gaps of a few steps. This is not included in ocf-data-sampler. We move this satellite interpolation into the app. This will work with both the legacy and new dataloader, and having this step in the app gives us the opportunity to take more care in the interpolation and monitor this step more closely

Closes #131

Checklist:

peterdudfield commented 3 days ago

Perhaps double checking it still works if there is no satellite data at all.

This is our back up where we just run tge ecmwf model only

dfulu commented 2 days ago

Perhaps double checking it still works if there is no satellite data at all.

This is our back up where we just run tge ecmwf model only

I've added a test for the case where there is no satellite data available