google-research / arco-era5

Recipes for reproducing Analysis-Ready & Cloud Optimized (ARCO) ERA5 datasets.
https://cloud.google.com/storage/docs/public-datasets/era5
Apache License 2.0
287 stars 22 forks source link

time unit error when opening single-level-forecast.zarr-v2 #59

Closed amarant closed 11 months ago

amarant commented 11 months ago

Hi, If I try to open the single-level-forecast.zarr-v2 with :

sl_forecasts = xr.open_zarr(
    'gs://gcp-public-data-arco-era5/co/single-level-forecast.zarr-v2/', 
    chunks={'time': 48},
    consolidated=True,
)

I get the error : ValueError: Failed to decode variable 'valid_time': unable to decode time units 'hours since 1900-01-01 06:00:00' with "calendar 'proleptic_gregorian'". Try opening your dataset with decode_times=False or installing cftime if it is not installed. But cftime is already installed.

Also the time unit seems strange : hours since 1900-01-01T06:00:00.000000, it is probably in days from a more recent date.

So adding decode_times=False and doing

units, reference_date = sl_forecasts.time.attrs['units'].split('since')
sl_forecasts['time'] = pd.date_range(start=reference_date, periods=sl_forecasts.sizes['time'], freq='H')

would be wrong.

Any idea how we can open the dataset with the correct time ?

Thanks.

DarshanSP19 commented 11 months ago

Hey @amarant Can you please try again? We have added a fix. Thanks.

amarant commented 11 months ago

It works now, thanks.

alxmrs commented 11 months ago

Wow, great work Sahil and Darshan!