Open SorooshMani-NOAA opened 2 weeks ago
Pinning numpy
to <2
and netCDF4
to <1.7
will resolve the issue:
In [1]: from netCDF4 import Dataset
In [2]: Dataset('https://icdc.cen.uni-hamburg.de/thredds/dodsC/ftpthredds/hamtide/k2.hamtide11a.nc')
Error:curl error: Problem with the SSL CA cert (path? access rights?)
curl error details:
Warning:oc_open: Could not read url
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
Cell In[2], line 1
----> 1 Dataset('https://icdc.cen.uni-hamburg.de/thredds/dodsC/ftpthredds/hamtide/k2.hamtide11a.nc')
File src/netCDF4/_netCDF4.pyx:2470, in netCDF4._netCDF4.Dataset.__init__()
File src/netCDF4/_netCDF4.pyx:2107, in netCDF4._netCDF4._ensure_nc_success()
OSError: [Errno -68] NetCDF: I/O failure: 'https://icdc.cen.uni-hamburg.de/thredds/dodsC/ftpthredds/hamtide/k2.hamtide11a.nc'
Ideally we should try to resolve this without the need to pin packages!
It's obvious, but just for the sake of completeness, if I try to read opendap netcdf through the xarray
it fails with version 1.7
and up.
Note that the version of numpy
above was required due to compatibility issue with netCDF-python. It seems numpy==2
doesn't work with older versions of netCDF4
(<1.7
)!
In [1]: from netCDF4 import Dataset
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[1], line 1
----> 1 from netCDF4 import Dataset
File ~/miniconda3/envs/adcircpy/lib/python3.10/site-packages/netCDF4/__init__.py:3
1 # init for netCDF4. package
2 # Docstring comes from extension module _netCDF4.
----> 3 from ._netCDF4 import *
4 # Need explicit imports for names beginning with underscores
5 from ._netCDF4 import __doc__
File src/netCDF4/_netCDF4.pyx:1, in init netCDF4._netCDF4()
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
If I install netCDF4
python package using conda/mamba I won't run into the original issue with Hamtide; I can read the URL. Maybe we need to just find the right way to install it using PIP, or just pin the version in the CI tests installation
This seems to be the solution! https://github.com/Unidata/netcdf4-python/issues/1179#issuecomment-1205668023
We need to install the netcdf library by rebuilding:
python -m pip install --upgrade --force-reinstall --no-deps --no-cache-dir netcdf4 --no-binary netcdf4
It seems that the updated netCDF4 package cannot work with the OPeNDAP link