blaylockbk / Herbie

Download numerical weather prediction datasets (HRRR, RAP, GFS, IFS, etc.) from NOMADS, NODD partners (Amazon, Google, Microsoft), ECMWF open data, and the University of Utah Pando Archive System.
https://herbie.readthedocs.io/
MIT License
492 stars 74 forks source link

ECMWF ensemble inconsistent dataset order #369

Open williamhobbs opened 1 month ago

williamhobbs commented 1 month ago

For some ECMWF ensemble fxx values, the first dataset is all members and the second is the mean, but for other fxx values, the order is swapped. Is this expected?

Started with https://herbie.readthedocs.io/en/latest/gallery/ecmwf_models/ecmwf.html#Ensemble-Forecast-Products.

Here's an example:

init_time = '2024-04-10 00:00'
fxx=15
ds = Herbie(date=init_time, model='ifs',product='enfo',fxx=fxx).xarray(":100[uv]:")

ds[0] returns all 50 members:

image

and ds[1] returns (presumably?) the mean:

image

But, if I change fxx, I sometimes get the opposite:

fxx=18
ds = Herbie(date=init_time, model='ifs',product='enfo',fxx=fxx).xarray(":100[uv]:")

image

williamhobbs commented 1 month ago

As a workaround to get just the dataset with all the members, I check to see which one has number in the dimensions:

if 'number' in list(ds[1].dims):
    ds = ds[1] 
else:
    ds = ds[0]

I'm sure there's a better way, but I think this works. And it could be extended a bit to re-order the datasets into a consistent order.