More training data from Open-Meteo

[patrick-zippenfenig]

Hi, author of Open-Meteo here.

I noticed you are using UKMO data for training, due to the 3 months limitation at Open-Meteo. I am working on an archive for high resolution models. Data is available from GFS from April 2021 and DWD ICON from November 2022.

The API endpoint is a bit slow at the moment, but performance and data availability will be improved in the next weeks.

Let me know if this helps or if you need any other data!

EDIT: Most data is also available as open-data through an AWS S3 sponsorship.

[peterdudfield]

Hi @patrick-zippenfenig

Thanks for getting in touch and thank you work on Open-Meteo.

We've decided to use Open-Meteo for running inference, as its super convenient to use. This means people who are using this forecast don't have to download large amount of NWP data.

We've trained our model using our own NWP dataset from UK MetOffice. We've also run validation of our models on ICON dataset which put on Hugging Face here. This is from 2020 onwards.

Good to know that the an archive can also been accessed through Open-Meteo, and we'll be sure to sign up if we start using this service.

Please do say if you would like us to use Open-Meteo for inference in a different way. Hopefully these two projects can work together rather than against each other

[patrick-zippenfenig]

Hi @peterdudfield,

thanks for the quick reply! I am glad Open-Meteo works great for your services. Yes, we can absolutely work together. I am happy to see open-source applications like PV forecasting based on open-data.

I did not know you already use ICON for training and also started archiving it. ICON is one of my preferred models and I can recommend using it.

Let me know if need any additional datasets in Open-Meteo. It is relatively straightforward to integrate new datasets like UKMO or satellite based radiation like SARAH. You can also reach out via info@open-meteo.com

Best, Patrick

[patrick-zippenfenig]

Quik update: UK Met Office data is now also available on Open-Meteo: https://open-meteo.com/en/docs/ukmo-api

Including data from March 2022 onwards: https://open-meteo.com/en/docs/historical-forecast-api#latitude=51.5085&longitude=-0.1257&start_date=2022-03-01&end_date=2022-12-31&models=ukmo_global_deterministic_10km,ukmo_uk_deterministic_2km

[peterdudfield]

Thanks for this @patrick-zippenfenig

Does this provide forecast at different time horizons, or is it jus the latest forecast made for each target time?

[patrick-zippenfenig]

I am collecting data for different lead times starting from last week: https://open-meteo.com/en/docs/previous-runs-api#latitude=51.5085&longitude=-0.1257&past_days=14&models=ukmo_global_deterministic_10km,ukmo_uk_deterministic_2km

Archived data only uses the latest forecast (smallest lead-time / horizon). It would be possible to download additional data and offer a time series for 24 hour or 48 hour lead times.

[peterdudfield]

Thanks for this @patrick-zippenfenig

[peterdudfield]

We could run inference using the uk metoffice now, ill create anew issue for this