try ICON data

[peterdudfield]

Would be great to try the model with ICON data,

1. Add option to switch to ICON data when loading data in inference.

This should be done here. You might have to play a bit with https://open-meteo.com/en/docs/dwd-api first, in order to get the right API call.

2. Also in need to do evaluation open on the model, use ICON data, https://huggingface.co/datasets/openclimatefix/dwd-icon-eu. Need to do #2 first

[peterdudfield]

Hey @EdFage Thanks for potentially doing this. If you just focus on 1., myself and @zakwatts might try and do 2, next week

[EdFage]

Hey @peterdudfield, will do. I've started and am making progress!

[peterdudfield]

Thanks @EdFage

[EdFage]

Hey @peterdudfield,

how do you want to implement this? Should i add an argument to the get_gfs_nwp() function so the user can choose ICON or GFS? or just add a variable inside get_gfs_nwp() so that the source isn't chosen by the user?

[peterdudfield]

Yea I would

• rename the function to just get_nwp
• add option in function and default to icon
• in the function have a different way of getting the data, depending if its gfs or icon

Thanks so much for your help

[EdFage]

Great.

I've got a problem: unless I'm missing something obvious the dwd api doesn't have a visibility variable. (https://open-meteo.com/en/docs/dwd-api)

I've tried doing inference with an xarray of everything except visibility but this gives me a runtime error of the form "regressor was trained on features {train_feature_set} != {feature_set}" from the psp package.

What would you suggest I do here?

[peterdudfield]

Interesting, could you add a dummy variable for the moment? See what on average it is in GFS, and set that in the pulled ICON data?

[EdFage]

yep will give that a go!

[peterdudfield]

Thank you

[peterdudfield]

Perhaps at 24000, and we just have to acknowlegde that it wont quite work here. Another option would be to put nans here, but im not quite sure what is best. I think the model can handle nans

[EdFage]

Not sure what you mean by 24,000? in any case, i've got things working using GFS visibility data and ICON for everything else. About to make a pull request

[peterdudfield]

Not sure what you mean by 24,000? in any case, i've got things working using GFS visibility data and ICON for everything else. About to make a pull request

ah sorry i wasnt clear. Perhaps the deafulty value could be 24K. I just got that from looking here - https://open-meteo.com/en/docs/gfs-api#hourly=visibility. But no worries if you have set it differently

[peterdudfield]

This is done in #15 , just need to update Readme.md

[peterdudfield]

@all-contributors please add @EdFage for code

[allcontributors[bot]]

@peterdudfield

I've put up a pull request to add @EdFage! :tada: