Should we track which model (and results) is being used

[peterdudfield]

Detailed Description

An idea to track which model is being used.

Context

• good to know how much this package is being used
• good to track results, and see how accurate they are

Possible Implementation

• save results to s3 bucket every time its run?
• other ideas?

[peterdudfield]

We could perhaps save these results to a HF repo

[Clemo97]

So @peterdudfield basically we track which of these models is used (Enphase, Solis, GivEnergy, and Solarman) ?

[peterdudfield]

yea could do those, also the different ml models used - https://github.com/openclimatefix/Open-Source-Quartz-Solar-Forecast/blob/main/quartz_solar_forecast/forecast.py#L111

[Clemo97]

Okay so track which inverters and models (gb and xgb) are used. Also what did you mean by HF repo ?

Since the models are being run locally, my thinking is that there needs to be code that is triggered when the run_forecast() method is run sending back data like model & inverter used and forecast data.

Any thoughts?

[peterdudfield]

yea exactly, so its thinking where do we send these meta data fo the forecast back to, This could be a hugging face repo, this could be and s3 bucket, but very open to ideas

[peterdudfield]

Okay so track which inverters and models (gb and xgb) are used. Also what did you mean by HF repo ?

Since the models are being run locally, my thinking is that there needs to be code that is triggered when the run_forecast() method is run sending back data like model & inverter used and forecast data.

Any thoughts?

gb and xgb are current different ml models

[peterdudfield]

This is now done in https://huggingface.co/openclimatefix/open-source-quartz-solar-forecast/tree/main/data/2024