Open mickaellalande opened 3 years ago
could you kindly share the full error traceback?
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-4-db5c6cefdcdc> in <module>
----> 1 xr.open_dataset(root_data_path+'/MODIS/MOD10C1_Terra/MOD10C1.A2000055.061.2020037182124.hdf')
~/miniconda3/envs/phd_v3/lib/python3.8/site-packages/xarray/backends/api.py in open_dataset(filename_or_obj, group, decode_cf, mask_and_scale, decode_times, autoclose, concat_characters, decode_coords, engine, chunks, lock, cache, drop_variables, backend_kwargs, use_cftime, decode_timedelta)
543
544 with close_on_error(store):
--> 545 ds = maybe_decode_store(store)
546
547 # Ensure source filename always stored in dataset object (GH issue #2550)
~/miniconda3/envs/phd_v3/lib/python3.8/site-packages/xarray/backends/api.py in maybe_decode_store(store, lock)
449
450 def maybe_decode_store(store, lock=False):
--> 451 ds = conventions.decode_cf(
452 store,
453 mask_and_scale=mask_and_scale,
~/miniconda3/envs/phd_v3/lib/python3.8/site-packages/xarray/conventions.py in decode_cf(obj, concat_characters, mask_and_scale, decode_times, decode_coords, drop_variables, use_cftime, decode_timedelta)
578 encoding = obj.encoding
579 elif isinstance(obj, AbstractDataStore):
--> 580 vars, attrs = obj.load()
581 extra_coords = set()
582 file_obj = obj
~/miniconda3/envs/phd_v3/lib/python3.8/site-packages/xarray/backends/common.py in load(self)
121 """
122 variables = FrozenDict(
--> 123 (_decode_variable_name(k), v) for k, v in self.get_variables().items()
124 )
125 attributes = FrozenDict(self.get_attrs())
~/miniconda3/envs/phd_v3/lib/python3.8/site-packages/xarray/backends/netCDF4_.py in get_variables(self)
397
398 def get_variables(self):
--> 399 dsvars = FrozenDict(
400 (k, self.open_store_variable(k, v)) for k, v in self.ds.variables.items()
401 )
~/miniconda3/envs/phd_v3/lib/python3.8/site-packages/xarray/core/utils.py in FrozenDict(*args, **kwargs)
436
437 def FrozenDict(*args, **kwargs) -> Frozen:
--> 438 return Frozen(dict(*args, **kwargs))
439
440
~/miniconda3/envs/phd_v3/lib/python3.8/site-packages/xarray/backends/netCDF4_.py in <genexpr>(.0)
398 def get_variables(self):
399 dsvars = FrozenDict(
--> 400 (k, self.open_store_variable(k, v)) for k, v in self.ds.variables.items()
401 )
402 return dsvars
~/miniconda3/envs/phd_v3/lib/python3.8/site-packages/xarray/backends/netCDF4_.py in open_store_variable(self, name, var)
374 # netCDF4 specific encoding; save _FillValue for later
375 encoding = {}
--> 376 filters = var.filters()
377 if filters is not None:
378 encoding.update(filters)
netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable.filters()
netCDF4/_netCDF4.pyx in netCDF4._netCDF4._ensure_nc_success()
RuntimeError: NetCDF: Attempting netcdf-4 operation on netcdf-3 file
OK, so the (misleading) error message here is ultimately coming from the underlying netCDF library, not Xarray.
Inspecting these files, it looks like they are HDF4 files, not netCDF3. We haven't really tested HDF4 files much with Xarray, but it would be nice to support them.
Late to the party. If this is still an issue for you @mickaellalande, MODIS data can also be read using engine="rasterio"
(needs rioxarray
).
In [1]: import xarray as xr
...: with xr.open_dataset('source/MODIS/MOD10C1.A2000055.061.2020037182124.hdf', engine="rasterio") as ds:
...: display(ds)
...:
<xarray.Dataset>
Dimensions: (band: 1, x: 7200, y: 3600)
Coordinates:
* band (band) int64 1
* x (x) float64 -180.0 -179.9 -179.9 ... 179.9 180.0
* y (y) float64 89.97 89.92 89.88 ... -89.93 -89.98
spatial_ref int64 ...
Data variables:
Day_CMG_Snow_Cover (band, y, x) float32 ...
Day_CMG_Clear_Index (band, y, x) float32 ...
Day_CMG_Cloud_Obscured (band, y, x) float32 ...
Snow_Spatial_QA (band, y, x) float32 ...
Attributes: (12/50)
ALGORITHMPACKAGEACCEPTANCEDATE: 09-2013
ALGORITHMPACKAGEMATURITYCODE: Normal
ALGORITHMPACKAGENAME: MOD_PR10C1
ALGORITHMPACKAGEVERSION: 6
ASSOCIATEDINSTRUMENTSHORTNAME.1: MODIS
ASSOCIATEDPLATFORMSHORTNAME.1: Terra
... ...
SNOWCOVERPERCENT: 28
SOUTHBOUNDINGCOORDINATE: -90.0
SPSOPARAMETERS: none
VERSIONID: 61
WESTBOUNDINGCOORDINATE: -180.0
ZONEIDENTIFIER: Other Grid System
I'm trying to open MODIS .hdf files, but I get the error :
NetCDF: Attempting netcdf-4 operation on netcdf-3 file
. Does anyone knows how to open that files? (https://nsidc.org/data/MOD10C1)I already opened hdf files from another product without any issue... (https://nsidc.org/data/MOD10CM)
Here are two examples, with one that works and the other one that causes the issue: MODIS.zip
Thanks in advance for your help!
Output of xr.show_versions()
INSTALLED VERSIONS ------------------ commit: None python: 3.8.5 | packaged by conda-forge | (default, Jul 24 2020, 01:25:15) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 4.19.0-16-amd64 machine: x86_64 processor: byteorder: little LC_ALL: None LANG: fr_FR.UTF-8 LOCALE: fr_FR.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.16.0 pandas: 1.1.0 numpy: 1.19.1 scipy: 1.5.2 netCDF4: 1.5.4 pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.4.0 cftime: 1.2.1 nc_time_axis: 1.2.0 PseudoNetCDF: None rasterio: 1.1.5 cfgrib: 0.9.8.5 iris: None bottleneck: None dask: 2.21.0 distributed: 2.21.0 matplotlib: 3.2.0 cartopy: 0.17.0 seaborn: None numbagg: None pint: None setuptools: 49.2.0.post20200712 pip: 20.2 conda: None pytest: 6.0.0 IPython: 7.16.1 sphinx: None