search

zarr-developers / geozarr-spec

This document aims to provides a geospatial extension to the Zarr specification. Zarr specifies a protocol and format used for storing Zarr arrays, while the present extension defines conventions and recommendations for storing multidimensional georeferenced grid of geospatial observations (including rasters).
106 stars 10 forks source link

netCDF4 interoperability #39

Open christine-e-smit opened 4 months ago

christine-e-smit commented 4 months ago

Using python NetCDF4 library version 1.6.2, I was able to load a zarr store.

Data used: https://github.com/zarr-developers/geozarr-spec/issues/36

The "netCDF-like" zarr store could be opened. The "geotiff-like" zarr store could not be opened.

To reproduce:

  1. Download and unzip the netCDF-like zarr store.
  2. Open in python using the nczarr syntax: 
    import netCDF4
nc = netCDF4.Dataset('file:///YOUR/PATH/TO/GLDAS_NOAH025_3H.zarr#mode=nczarr,zarr')
christine-e-smit commented 4 months ago

@ethanrd - any idea why the "geotiff-like" zarr store caused python to crash for me? I looked at the metadata, I can't immediately see what the issue is. It does load just fine with xarray.

>>> import netCDF4
>>> netCDF4.Dataset('file:///PATH/planet-fusion.zarr#mode=nczarr,zarr')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "src/netCDF4/_netCDF4.pyx", line 2469, in netCDF4._netCDF4.Dataset.__init__
  File "src/netCDF4/_netCDF4.pyx", line 2028, in netCDF4._netCDF4._ensure_nc_success
OSError: [Errno -137] NetCDF: NCZarr error: 'file:///PATH/planet-fusion.zarr#mode=nczarr,zarr'
christine-e-smit commented 4 months ago

@ethanrd - Can I create a new zarr store using the netCDF4 library? The documentation for the Dataset class (https://unidata.github.io/netcdf4-python/#netCDF4.Dataset) indicates that you need to specify the data format, but I don't see anything with 'zarr' as an option. Just because, I tried NC_ZARR and NCZARR, but neither of those worked.

christine-e-smit commented 4 months ago

@ethanrd - One more question! It looks as though the NetCDF library is having trouble with compression.

import netCDF4
nc = netCDF4.Dataset("file:///YOUR/PATH/GLDAS_NOAH025_3H.zarr#mode=nczarr,zarr")
nc["Tair_f_inst"][0][401][238]

gives the error:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[33], line 1
----> 1 nc["Tair_f_inst"][0][401][238]

File src[/netCDF4/_netCDF4.pyx:4972](http://localhost:8888/netCDF4/_netCDF4.pyx#line=4971), in netCDF4._netCDF4.Variable.__getitem__()

File src[/netCDF4/_netCDF4.pyx:5930](http://localhost:8888/netCDF4/_netCDF4.pyx#line=5929), in netCDF4._netCDF4.Variable._get()

File src[/netCDF4/_netCDF4.pyx:2034](http://localhost:8888/netCDF4/_netCDF4.pyx#line=2033), in netCDF4._netCDF4._ensure_nc_success()

RuntimeError: NetCDF: Filter error: undefined filter encountered

I'm almost certain that I talked with someone at Unidata about this more than a year ago, but I can't find where I had the conversation.