search

Unidata / netcdf4-python

netcdf4-python: python/numpy interface to the netCDF C library
http://unidata.github.io/netcdf4-python
MIT License
757 stars 265 forks source link

bug reading netcdf via opendap #753

Open jhlasheras opened 6 years ago

jhlasheras commented 6 years ago

I found, what i guess is a bug in the netcdf4 library.

I am using anaconda with python 3.6.3 and Ipython 6.1.0

I tried to read a file via opendap when just starting my ipython, which works:

import netCDF4 ds = netCDF4.Dataset('http://thredds.socib.es/thredds/dodsC/hf_radar/hf_radar_ibiza-scb_codarssproc001/L1/2016/dep0001_hf-radar-ibiza_scb-codarssproc001_L1_2016-02.nc')

The problem arises when I load some other libraries and try to load the same file. Doing the following:

import numpy as np import pandas as pd import matplotlib.pyplot as plt

ds = netCDF4.Dataset('http://thredds.socib.es/thredds/dodsC/hf_radar/hf_radar_ibiza-scb_codarssproc001/L1/2016/dep0001_hf-radar-ibiza_scb-codarssproc001_L1_2016-02.nc')

OSError: [Errno -36] NetCDF: Invalid argument: b'http://thredds.socib.es/thredds/dodsC/hf_radar/hf_radar_ibiza-scb_codarssproc001/L1/2016/dep0001_hf-radar-ibiza_scb-codarssproc001_L1_2016-02.nc'

However, I can read local netcdf files and also files via opendap with a shorter name:

ds2 = netCDF4.Dataset('http://thredds.socib.es/thredds/dodsC/operational_models/oceanographical/hydrodynamics/wmop_surface/latest.nc')

This works fine! It also works fine when executing the same code in jupyter notebook. Any explanation? Cheers

jswhit commented 6 years ago

Sorry, but I have no idea. Does it work if you don't import pandas, or if you import netCDF4 last?

rsignell-usgs commented 6 years ago

The provided example:

import netCDF4
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

ds = netCDF4.Dataset('http://thredds.socib.es/thredds/dodsC/hf_radar/hf_radar_ibiza-scb_codarssproc001/L1/2016/dep0001_hf-radar-ibiza_scb-codarssproc001_L1_2016-02.nc')

works fine for me, produces no errors.

My NetCDF version is 1.3.1

jswhit commented 6 years ago

Works here too.

jhlasheras commented 6 years ago

That's rigth. I have tried it in a macbook and it works fine. However in my ubuntu 14.04 it continuosly give an error. Both of them using netCDF4 version 1.3.1. Also in other mates computer with similar configuration the same error happened.

May anaconda netcdf library be interfering with some other netcdf library?

dopplershift commented 6 years ago

This might depend on the version of the netCDF-C library being used. IIRC, there were some versions that got a bit more picky about some of the opendap parsing.

cgohlke commented 6 years ago

At least on Windows, this issue is fixed by https://github.com/Unidata/netcdf-c/pull/366 (for me)

jhlasheras commented 6 years ago

I found the problem may be due to the matplotlib backend. When initializing with Tk backend the problem dissaspears.

Adding this at the begining of the script solves the problem:

import matplotlib matplotlib.use('TkAgg') import matplotlib.pyplot as plt

everything works fine untill I execute %matplotlib