Can extract some but not other variable fields from a NetCDF file

[GoogleCodeExporter]

What steps will reproduce the problem?

>>> import numpy
>>> from netCDF4 import Dataset
>>> infile = Dataset('g01l01s01.nc','r')
>>> infile.variables
OrderedDict([(u'z', <netCDF4.Variable object at 0x195d0ed0>), (u'Prz', 
<netCDF4.Variable object at 0x195d6750>), (u'T1', <netCDF4.Variable object at 
0x195d6cd0>), (u'T2', <netCDF4.Variable object at 0x195d6ed0>), (u'Sal', 
<netCDF4.Variable object at 0x195db7d0>), (u'Cdt1', <netCDF4.Variable object at 
0x195db150>), (u'Cdt2', <netCDF4.Variable object at 0x195db3d0>), (u'Dst', 
<netCDF4.Variable object at 0x195db850>), (u'Oxy1', <netCDF4.Variable object at 
0x195db8d0>), (u'Oxy2', <netCDF4.Variable object at 0x195db950>), (u'Xmi', 
<netCDF4.Variable object at 0x195db9d0>), (u'Flu', <netCDF4.Variable object at 
0x195e5ad0>), (u'Par', <netCDF4.Variable object at 0x195e5350>), (u'Rho', 
<netCDF4.Variable object at 0x195e57d0>), (u'Sigt', <netCDF4.Variable object at 
0x195e5850>), (u'Des', <netCDF4.Variable object at 0x195e5b50>), (u'Gpa', 
<netCDF4.Variable object at 0x195e5bd0>), (u'PoT', <netCDF4.Variable object at 
0x195e5c50>), (u'profile', <netCDF4.Variable object at 0x195e5cd0>)])
>>> z = infile.variables['z'][:]
>>> z
array([   2. ,    2.5,    3. , ...,  831. ,  831.5,  832. ], dtype=float32)
>>> Prz = infile.variables['Prz'][:]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "netCDF4.pyx", line 2692, in netCDF4.Variable.__getitem__ (netCDF4.c:33372)
TypeError: only integer arrays with one element can be converted to an index

What is the expected output? What do you see instead?

I can successfully read the values from the z variable in the NetCDF file, but 
I get the error message shown when I attempt to read Prz or any of the other 
non-z variables in the NetCDF file.  I can't see any differences between z and 
Prz that would cause this.  They're both float32 and have the same shape, and I 
can see that the numbers are there using both ncdump and ncks.  I'm at a loss 
as to why I can't extract Prz etc. using netcdf4-python.

What version of the product are you using? On what operating system?

I'm using the version pulled down via svn on today, i.e. 4/24/13.
I'm using this with Python 2.7, NetCDF 4.1.1 and HDF 1.8.9 on a Linux system 
running kernel 2.6.18-308.11.1.el5.

Please provide any additional information below.

The NetCDF file is attached.

Original issue reported on code.google.com by steven.k...@gmail.com on 24 Apr 2013 at 9:21

Attachments:

• g01l01s01.nc

[GoogleCodeExporter]

The problem is that the variable has scale_factor and add_offset attributes, 
but they are set to null strings.  netcdf4-python tries to apply the scale and 
offset to the variable, and ends up multiplying an array by a null string, 
which doesn't work.  You can turn this auto-scaling off and it works (see 
below).  netcdf4-python could check to see if scale_factor and add_offset are 
valid before trying to do the re-scaling, but the fundamental problem here is a 
malformed netcdf file.

from netCDF4 import Dataset
f = Dataset('g01l01s01.nc')
v = f.variables['Prz']
v.set_auto_maskandscale(False)
print v[:]
f.close()

Original comment by whitaker.jeffrey@gmail.com on 24 Apr 2013 at 10:06

[GoogleCodeExporter]

I've added code to check that the scale_factor and add_offset attributes can be 
converted to floats, if they cannot a warning is issued and the the data is not 
unpacked.

Original comment by whitaker.jeffrey@gmail.com on 24 Apr 2013 at 10:23

[GoogleCodeExporter]

Thank you very much.  I had some vague notion that the scale_factor might have 
something to do with it, and that's not the only way that particular NetCDF 
file is malformed.  I encountered this difficulty while working with the file's 
originator to attempt to clean it up.  Thanks again!

Original comment by steven.k...@gmail.com on 25 Apr 2013 at 3:17

[GoogleCodeExporter]

Original comment by whitaker.jeffrey@gmail.com on 26 Feb 2014 at 2:04

• Changed state: Fixed