openclimatefix / uk-wind-modelling-basic

Simple wind forecast repo
3 stars 1 forks source link

Retrieving wind speed data #1

Open srennug opened 1 year ago

srennug commented 1 year ago

Describe the bug

When pulling a larger time frame of wind speed data, errors occur. It also takes a long time to retrieve the data, not sure if this can be improved

To Reproduce

Set periods > 400

Expected behaviour

This should just pull all the values for u v wind speeds from the selected time frame.

Additional context

Add any other context about the problem here. windspeeds.txt

peterdudfield commented 1 year ago

There might be a hard limit on number of periods.

Perhaps you could do a limit of 100, and then loop through and concatenate the data together

srennug commented 1 year ago

windspeeds_2.txt

Attached is the code I've been using to do this but I'm still getting the same error.

EOFError: End of file: <_io.BufferedReader name='/Users/tomelder/data/gfs/20220506/subset_f5effba6__gfs.t00z.pgrb2.0p25.f000'>

It seems to work until it reaches a certain date. The code from the previous message could take 200 period from the start of 2022 but not 2021.

peterdudfield commented 1 year ago

hmm, what date does it fail on?

You might need to H.download(searchString, overwrite=True) or something like that?

srennug commented 1 year ago

The first two months of 2021 and March of 2022 give errors. I was able to take a year of values from 2021.03.01 at 24hr intervals, but it gives errors when I try to take 12hr intervals. I think the errors are frequent throughout the data set.

I've written a function which replaces the dates that give an error with the previous u v wind speeds.

peterdudfield commented 1 year ago

Great work. I would probably leave the blank data blank, so we dont end up with any weird results in the end. Good work getting all that data

peterdudfield commented 1 year ago

Ok, to close this now? Did this get solved?