Deltares / hydromt

HydroMT: Automated and reproducible model building and analysis
https://deltares.github.io/hydromt/
MIT License
68 stars 28 forks source link

error with ERA5 zarr conversion in deltares_data (?) #931

Open atsiokanos opened 3 months ago

atsiokanos commented 3 months ago

HydroMT version checks

Reproducible Example

dc = DataCatalog(data_libs="deltares_data")

era5_daily = dc.get_rasterdataset("era5_daily", bbox=[0.259, 6.234, 5.542, 10.378], time_tuple=(datetime.strptime("2022-01-01", '%Y-%m-%d'), datetime.strptime("2022-05-01", '%Y-%m-%d')))

era5_daily_zarr = dc.get_rasterdataset("era5_daily_zarr", bbox=[0.259, 6.234, 5.542, 10.378], time_tuple=(datetime.strptime("2022-01-01", '%Y-%m-%d'), datetime.strptime("2022-05-01", '%Y-%m-%d')))

era5_daily.sel(longitude=2.25, latitude= 8.75)["precip"].plot() era5_daily_zarr.sel(longitude=2.25, latitude= 8.75)["precip"].plot()

Current behaviour

It appears there is an issue with converting ERA5 data to zarr format in our catalogs. The ERA5 zarr data show values exceeding 900 mm/day precipitation for an extended period (see the differences between era5_daily and era5_daily_zarr in the provided code example between mid Feb and March).

I'm not entirely sure if here is the most appropriate place to report this issue, but since we use the DD catalog, it is worth exploring.

Desired behaviour

make them identical

Additional context

No response

DirkEilander commented 3 months ago

@aweerts have you seen this before?