openclimatefix / ocf-ml-metrics

Collection of simple baseline models and metrics for standardized evaluation of OCF forecasting models
MIT License
4 stars 0 forks source link

Added support for calculating common metrics for 2d inputs #13

Closed confusedmatrix closed 9 months ago

confusedmatrix commented 9 months ago

Pull Request

Description

During training over batches of data on PVNet, I was receiving errors due to common_metrics returning a float for each metric (RMSE & MAE). For multiple forecast horizons (e.g. every 30 minutes), we need common_metrics to return a 1-d array, so this PR enables support for checking the input predictions array to common_metrics and calculates the mean appropriately depending on the dimensions of the input.

Tests have been updated to include 1-d and 2-d input expectations.

Fixes #

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

If your changes affect data processing, have you plotted any changes? i.e. have you done a quick sanity check?

Checklist:

codecov[bot] commented 9 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (294c3d5) 96.07% compared to head (ef50efa) 96.17%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #13 +/- ## ========================================== + Coverage 96.07% 96.17% +0.09% ========================================== Files 5 5 Lines 153 157 +4 ========================================== + Hits 147 151 +4 Misses 6 6 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.