Failure to load OPeNDAP in Colab

[giovaniceotto]

Recently, reading OPeNDAP files when running notebooks on Google Colab resutss in an OSError: [Errno -68] NetCDF: I/O failure

Relevant info:

• netCDF4 version: 1.6.0
• Python version: Python 3.7.13
• Environment: Google Colab

Example code:

import netCDF4

ds = netCDF4.Dataset('http://thredds.socib.es/thredds/dodsC/hf_radar/hf_radar_ibiza-scb_codarssproc001/L1/2016/dep0001_hf-radar-ibiza_scb-codarssproc001_L1_2016-02.nc')

Error:

Colab Notebook link: https://colab.research.google.com/drive/17juZulxoz8sahoDJVuU4uOMSnBC_xXDA?usp=sharing

[giovaniceotto]

Additional information: When using "netCDF4<1.6.0", the code runs fine, without any errors.

[Zeitsperre]

I've run into this issue as well:

Additionally, it looks as though this issue only happens when in a pure python environment running netCDF4==1.6.0. My test ensemble based on Anaconda and netCDF4==1.6.0 seems to be opening this OPeNDAP URL perfectly fine!

[Zeitsperre]

Update: I think the wheel being supplied via PyPI is broken. After compiling netCDF4==1.6.0 from sources, everything seems to be working perfectly fine. I needed to run the following after other dependencies were installed in my environment.

$ python -m pip install --upgrade --force-reinstall --no-deps --no-cache-dir netcdf4 --no-binary netcdf4

You may want to consider rebuilding your wheels (for manylinux * x86_64, anyway).

For reference: https://github.com/CSHS-CWRA/RavenPy/pull/193/commits/5bab92e26f3abbae7e54651bf67685cdce89c208

[jswhit]

You may want to consider rebuilding your wheels (for manylinux * x86_64, anyway).

What specifically do you think needs to be done differently when building the manylinux wheels?

[ocefpaf]

BTW, the screenshots in https://github.com/Unidata/netcdf4-python/issues/1179#issuecomment-1202720571 make it hard to read about the real issue. It seems that the problem is with curl and some SSL certificates. I get this when I try it locally:

Error:curl error: Problem with the SSL CA cert (path? access rights?)
curl error details: 
Warning:oc_open: Could not read url
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "src/netCDF4/_netCDF4.pyx", line 2353, in netCDF4._netCDF4.Dataset.__init__
  File "src/netCDF4/_netCDF4.pyx", line 1963, in netCDF4._netCDF4._ensure_nc_success

@jswhit what version/build of curl are you using in the wheel?

[jswhit]

curl i s 7.75.0, openssl is 1.0.2u

[ocefpaf]

It may be a version problem or something with the wheel build. I guess the only way to figure this out is to add an OPeNDAP test during the wheel test phase. This is how I reproduced it locally in case you want to check it out:

conda create --name TEST python=3 pip
pip install netcdf4

python -c "import netCDF4; ds = netCDF4.Dataset('http://thredds.socib.es/thredds/dodsC/hf_radar/hf_radar_ibiza-scb_codarssproc001/L1/2016/dep0001_hf-radar-ibiza_scb-codarssproc001_L1_2016-02.nc')"

Error:curl error: Problem with the SSL CA cert (path? access rights?)
curl error details: 
Warning:oc_open: Could not read url
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "src/netCDF4/_netCDF4.pyx", line 2353, in netCDF4._netCDF4.Dataset.__init__
  File "src/netCDF4/_netCDF4.pyx", line 1963, in netCDF4._netCDF4._ensure_nc_success
OSError: [Errno -68] NetCDF: I/O failure: b'http://thredds.socib.es/thredds/dodsC/hf_radar/hf_radar_ibiza-scb_codarssproc001/L1/2016/dep0001_hf-radar-ibiza_scb-codarssproc001_L1_2016-02.nc'

Then, if I do a conda install netcdf4, the test pass and I have these packages installed:

  Name       Version  Build                    Channel    
──────────────────────────────────────────────────────────
  curl       7.83.1   h2283fc2_0               conda-forge
  hdf5       1.12.2   nompi_h4df4325_100       conda-forge
  libcurl    7.83.1   h2283fc2_0               conda-forge
  libnetcdf  4.8.1    nompi_h21705cb_103       conda-forge
  netcdf4    1.6.0    nompi_py310h9fd08d4_101  conda-forge
  openssl    3.0.5    h166bdaf_1               conda-forge

[ngam]

@ocefpaf @jswhit any way we could help here? FWIW, this issue is not present on M1 Macs.

[ocefpaf]

I believe that the wheel should be rebuilt. One test folks could try is to conda install it on colab and see if that works.

[barronh]

I was having the same problem. I installed once with "%pip install netcdf4<1.6" and once with "%pip install netcdf4<1.6". I can confirm it works with one, but not the other.

For each, I used ldd to look at the linked libraries. Below are two lists. The first is the full list where I have sorted to align libraries. The second is just the places where version number or library has changed. I'm not sure if this helps, but I am hoping someone who knows more about this would have some thoughts.

--- v15.txt 2022-09-19 17:49:58.161994184 +0000
+++ v16.txt 2022-09-19 17:49:59.071994132 +0000
@@ -1,18 +1,20 @@
-   linux-vdso.so.1 (0x00007ffecfb0d000)
+   linux-vdso.so.1 (0x00007ffdd1397000)
-   /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4 (0x00007f99aaa79000)
+   /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4 (0x00007fddc9e71000)
-   libnetcdf-cb1b5117.so.18.0.0 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libnetcdf-cb1b5117.so.18.0.0 (0x00007f99aa8bf000)
+   libnetcdf-0ef85704.so.19 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libnetcdf-0ef85704.so.19 (0x00007fddc9b8c000)
-   libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f99aa6a0000)
+   libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fddc996d000)
-   libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f99aa2af000)
+   libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fddc957c000)
-   libunwind.so.8 => /usr/lib/x86_64-linux-gnu/libunwind.so.8 (0x00007f99aa094000)
+   libunwind.so.8 => /usr/lib/x86_64-linux-gnu/libunwind.so.8 (0x00007fddc9361000)
-   libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f99a9d0b000)
+   libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fddc8fd8000)
-   libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f99a996d000)
+   libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fddc8c3a000)
-   libhdf5_hl-f3ea9bc7.so.200.0.0 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libhdf5_hl-f3ea9bc7.so.200.0.0 (0x00007f99aad38000)
+   libhdf5_hl-f45ebdf9.so.200.1.0 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libhdf5_hl-f45ebdf9.so.200.1.0 (0x00007fddca12c000)
-   libhdf5-3e8ef4e5.so.200.0.0 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libhdf5-3e8ef4e5.so.200.0.0 (0x00007f99a9382000)
+   libhdf5-115e850b.so.200.2.0 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libhdf5-115e850b.so.200.2.0 (0x00007fddc85cc000)
-   libsz-57467d8a.so.2.0.1 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libsz-57467d8a.so.2.0.1 (0x00007f99aad31000)
+   libsz-57467d8a.so.2.0.1 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libsz-57467d8a.so.2.0.1 (0x00007fddca125000)
-   libaec-e300f322.so.0.0.10 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libaec-e300f322.so.0.0.10 (0x00007f99aad25000)
+   libaec-e300f322.so.0.0.12 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libaec-e300f322.so.0.0.12 (0x00007fddca115000)
-   libcurl-8c767087.so.4.7.0 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libcurl-8c767087.so.4.7.0 (0x00007f99a9095000)
+   libcurl-8c767087.so.4.7.0 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libcurl-8c767087.so.4.7.0 (0x00007fddc7e33000)
-   libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f99a8e78000)
+   libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fddc83af000)
+   libzstd-c4cc7033.so.1.5.2 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libzstd-c4cc7033.so.1.5.2 (0x00007fddc8120000)
+   libblosc-a9e8a7c7.so.1.21.1 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libblosc-a9e8a7c7.so.1.21.1 (0x00007fddc8289000)
-   libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f99a8c74000)
+   libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fddc8a36000)
-   /lib64/ld-linux-x86-64.so.2 (0x00007f99aace9000)
+   /lib64/ld-linux-x86-64.so.2 (0x00007fddca0e1000)
-   liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f99a8a4e000)
+   liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007fddc79f5000)
-   libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f99a8836000)
+   libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fddc7c1b000)

Below is a more narrow list of just where versions have changed.

-   libnetcdf-cb1b5117.so.18.0.0 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libnetcdf-cb1b5117.so.18.0.0 (0x00007f99aa8bf000)
+   libnetcdf-0ef85704.so.19 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libnetcdf-0ef85704.so.19 (0x00007fddc9b8c000)
-   libhdf5_hl-f3ea9bc7.so.200.0.0 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libhdf5_hl-f3ea9bc7.so.200.0.0 (0x00007f99aad38000)
+   libhdf5_hl-f45ebdf9.so.200.1.0 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libhdf5_hl-f45ebdf9.so.200.1.0 (0x00007fddca12c000)
-   libhdf5-3e8ef4e5.so.200.0.0 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libhdf5-3e8ef4e5.so.200.0.0 (0x00007f99a9382000)
+   libhdf5-115e850b.so.200.2.0 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libhdf5-115e850b.so.200.2.0 (0x00007fddc85cc000)
-   libaec-e300f322.so.0.0.10 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libaec-e300f322.so.0.0.10 (0x00007f99aad25000)
+   libaec-e300f322.so.0.0.12 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libaec-e300f322.so.0.0.12 (0x00007fddca115000)
+   libzstd-c4cc7033.so.1.5.2 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libzstd-c4cc7033.so.1.5.2 (0x00007fddc8120000)
+   libblosc-a9e8a7c7.so.1.21.1 => /usr/local/lib/python3.7/dist-packages/netCDF4/../netCDF4.libs/libblosc-a9e8a7c7.so.1.21.1 (0x00007fddc8289000)

[Zeitsperre]

The newest release (1.6.1) hasn't fixed the problems for my use case either:

src/netCDF4/_netCDF4.pyx:2463: in netCDF4._netCDF4.Dataset.__init__
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   OSError: [Errno -68] NetCDF: I/O failure: b'https://pavics.ouranos.ca/twitcher/ows/proxy/thredds/dodsC/birdhouse/disk2/testdata/raven/raven-gr4j-cemaneige/Salmon-River-Near-Prince-George_meteo_daily.nc'

src/netCDF4/_netCDF4.pyx:2026: OSError
----------------------------- Captured stderr call -----------------------------
Error:curl error: Problem with the SSL CA cert (path? access rights?)
curl error details: 
Warning:oc_open: Could not read url

[giovaniceotto]

@jswhit, I am available to do some digging and try to solve this issue since it is causing a lot of problem for other libraries. Do you have any suggestions to where should I start looking?

[ocefpaf]

This is not a solution, just a work around, but you can try:

!pip install -q condacolab
import condacolab
condacolab.install()

import condacolab
condacolab.check()

!mamba install netcdf4

and then,

import netCDF4

ds = netCDF4.Dataset('http://thredds.socib.es/thredds/dodsC/hf_radar/hf_radar_ibiza-scb_codarssproc001/L1/2016/dep0001_hf-radar-ibiza_scb-codarssproc001_L1_2016-02.nc')

I'll check the openssl version used and try to add a regression test in the wheel building process to investigate. I can confirm that this doesn't happen in the Windows wheel and @ngam confirmed it doesn't happen in the M1 wheel. It must be the Linux openssl version then.

[ebettenc]

In the Aug 8 traceback above (and pasted below), the URL string gets turned into a byte string by netCDF4. Could that be relevant? I had this problem and fixed it by using an older version of netCDF4 that keeps the string as a string. There was a similar issue with strings converted to byte strings 2 years ago: https://github.com/pydata/xarray/issues/4859

Traceback (most recent call last): File "", line 1, in File "src/netCDF4/_netCDF4.pyx", line 2353, in netCDF4._netCDF4.Dataset.init File "src/netCDF4/_netCDF4.pyx", line 1963, in netCDF4._netCDF4._ensure_nc_success OSError: [Errno -68] NetCDF: I/O failure: b'http://thredds.socib.es/thredds/dodsC/hf_radar/hf_radar_ibiza-scb_codarssproc001/L1/2016/dep0001_hf-radar-ibiza_scb-codarssproc001_L1_2016-02.nc'

[bertcoerver]

Still having this problem with netcdf4==1.6.3.

[Gui-FernandesBR]

I just used the colab notebook pinned in the first comment and used the new netcdf4==1.6.4 version. Everything is running as expected,

I think this issue was solved.

See https://github.com/Unidata/netcdf4-python/issues/1246#issuecomment-1577568553

[ocefpaf]

There are a few overlapping issues that may be closed. Posting them here to facilitate closing later:

https://github.com/Unidata/netcdf4-python/issues/1199 https://github.com/Unidata/netcdf4-python/issues/812 https://github.com/Unidata/netcdf4-python/issues/753