openclimatefix / ocf_datapipes

OCF's DataPipe based dataloader for training and inference
MIT License
13 stars 11 forks source link

Add loading wind power and India PV Power #260

Closed jacobbieker closed 6 months ago

jacobbieker commented 6 months ago

Pull Request

Description

Adds loading wind power, such as for India. Also adds a test for loading India PV power

Fixes #

How Has This Been Tested?

Unit Tests

Checklist:

codecov[bot] commented 6 months ago

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (e86aabf) 77.51% compared to head (0972bb7) 77.99%.

Files Patch % Lines
ocf_datapipes/load/wind/utils.py 90.90% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #260 +/- ## ========================================== + Coverage 77.51% 77.99% +0.47% ========================================== Files 118 120 +2 Lines 5364 5457 +93 ========================================== + Hits 4158 4256 +98 + Misses 1206 1201 -5 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

jacobbieker commented 6 months ago

Thanks! Yeah, for the failing test, in discussion here https://github.com/openclimatefix/india-forecast-app/issues/2 as to what to do about it, as its from nowcasting_datamodel's check of PV data

peterdudfield commented 6 months ago

Could you keep the validator there for PV, but not for Wind?

jacobbieker commented 6 months ago

The validator isn't running for Wind, but is for PV. We can add it back for PV, but we probably should put the definition of the names in datapipes, not in datamodel. As of right now, the validator validates passiv and pvoutput.org from nowcasting_datamodels, and adds SV here. So I can re-add it, but would like to remove the dependency on nowcasting_datamodel

peterdudfield commented 6 months ago

The validator isn't running for Wind, but is for PV. We can add it back for PV, but we probably should put the definition of the names in datapipes, not in datamodel. As of right now, the validator validates passiv and pvoutput.org from nowcasting_datamodels, and adds SV here. So I can re-add it, but would like to remove the dependency on nowcasting_datamodel

I agree, yea nice