jswhit
commented
5 years ago ncdump gives me the same error (with version 4.6.3), so it's definitely not an issue with netcdf4-python.
Closed rabernat closed 5 years ago
jswhit
commented
5 years ago ncdump gives me the same error (with version 4.6.3), so it's definitely not an issue with netcdf4-python.
jswhit
commented
5 years ago Seems to be an issue with the opendap server configuration - if I download the netcdf file to disk I can open it and read the data just fine.
jswhit
commented
5 years ago Closing this - if it does turn out to be a netcdf4-python issue feel free to reopen.
kwilcox
commented
5 years ago @lesserwhirls I’ve seen this before but can’t recall the issue or fix. I believe it was a THREDDS thing. Anything seem familiar?
lesserwhirls
commented
5 years ago If I recall, the issue was that netCDF-C started being strict about _FillValue type matching variable type starting in version 4.6.2. This is fine, except for the case of reading some kinds of data through the TDS OPeNDAP endpoint for types not supported by the DAP2 data model. This should be fixed in version netCDF-C 4.6.3 and above. See https://github.com/Unidata/netcdf-c/issues/1316
kwilcox
commented
5 years ago Thanks! @rabernat can you try with an updated libnetcdf?
rabernat
commented
5 years ago Unfortunately trying to update libnetcdf gets me into dependency probems with gdal / rasterio
$ conda install -c conda-forge libnetcdf=4.7.0
Fetching package metadata ...........
Solving package specifications: .
UnsatisfiableError: The following specifications were found to be in conflict:
- libnetcdf 4.7.0*
- rasterio -> gdal 1.11.2 -> hdf5 1.8.14
- rasterio -> gdal 1.11.2 -> libnetcdf 4.3.2 -> curl 7.38.0 -> openssl 1.0.1*
Use "conda info <package>" to see the dependencies for each package.So it looks like I can't upgrate netcdf without breaking rasterio / gdal. Unfortunately I will not have time to try to resolve this soon.
rsignell-usgs
commented
4 years ago I can reproduce this problem with updated netcdf libs:
# Name Version Build Channel
h5netcdf 0.8.0 py_0 conda-forge
libnetcdf 4.7.3 nompi_h9f9fd6a_101 conda-forge
netcdf4 1.5.3 nompi_py37hd35fb8e_102 conda-forgeI can work around it in netcdf4 by the prepending the URL with the '[FillMismatch]' kludge:
import netCDF4
url = 'http://gliders.ioos.us/thredds/dodsC/deployments/drudnick/sp062-20190201T1350/sp062-20190201T1350.nc3.nc'
netCDF4.Dataset('[FillMismatch]'+url)
<class 'netCDF4._netCDF4.Dataset'>
root group (NETCDF3_CLASSIC data model, file format DAP2):
acknowledgment: This deployment supported by NOAA.
cdm_data_type: TrajectoryProfile
...but this doesn't work in xarray:
import xarray as xr
xr.open_dataset('[FillMismatch]'+url)
FileNotFoundError: [Errno 2] No such file or directory: b'/home/jovyan/hurricane-ike-water-levels/[FillMismatch]http:/gliders.ioos.us/thredds/dodsC/deployments/drudnick/sp062-20190201T1350/sp062-20190201T1350.nc3.nc'because xarray (not too surprisingly) thinks it's a local file.
tompc35
commented
4 years ago Ran into the same problem here with with libnetcdf version 4.7.1 and netcdf4 version 1.5.3.
Appending #fillmismatch to the url worked in xarray (version 0.15.0).
import xarray as xr
url = 'http://gliders.ioos.us/thredds/dodsC/deployments/drudnick/sp062-20190201T1350/sp062-20190201T1350.nc3.nc'
ds = xr.open_dataset(url+'#fillmismatch')
I am having trouble opening a dataset from a thredds opendap server. I am raising the issue here, well aware that this issue may belong elsewhere. But this has been a useful place to start.
The data is from the OOI gliders DAC: https://gliders.ioos.us/data/
Here is the opendap file: https://gliders.ioos.us/thredds/dodsC/deployments/drudnick/sp062-20190201T1350/sp062-20190201T1350.nc3.nc.html
gives
Is this a problem with the dataset? If so, the entirety of the OOI glider data is incompatible with the netCDF C library.
netcdf4 lib version: '4.6.2 of Dec 17 2018 19:36:03 $' netcdf4-python version: '1.5.0.1'
cc @kwilcox, @rsignell-USGS