Aritmetically broadcasting dataset variables?

[GoogleCodeExporter]

Hi,

let me try to explain with example:
========================================
>>> import numpy as np
>>> from netCDF4 import Dataset
>>> nc = Dataset('test.nc')
>>> lat = nc.variables['lat']
>>> lat.shape
(64,)
>>> lat[(lat > 35) & (lat < 45)]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "netCDF4.pyx", line 2637, in netCDF4.Variable.__getitem__ (netCDF4.c:30305)
  File "netCDF4.pyx", line 3110, in netCDF4.Variable._get (netCDF4.c:35744)
RuntimeError: NetCDF: Start+count exceeds dimension bound
>>> nplat = np.array(lat)
>>> nplat[(nplat > 10) & (nplat < 50)]
array([ 48.83524097,  46.04472663,  43.25419467,  40.46364818,
        37.67308963,  34.88252099,  32.09194388,  29.30135962,
        26.51076933,  23.72017393,  20.92957425,  18.13897099,
        15.34836476,  12.55775612])
========================================

So is there protocol limitation in implementing this kind of slicing while 
referencing netCDF dataset?

Original issue reported on code.google.com by klo...@gmail.com on 20 Jun 2013 at 11:12

[GoogleCodeExporter]

I think you want

lat[(lat[:] > 35) & (lat[:] < 45)]

which is equivalent to

lat[(np.asarray(lat) > 35) & (np.asarray(lat) < 45)]

You have to slice the variable first to read the data.  

lat > 35 just evaluates to True.  I suppose I could override the __gt__ 
operator to first read the data and then do the comparison - I'll think more on 
that.

Original comment by whitaker.jeffrey@gmail.com on 21 Jun 2013 at 2:42

[GoogleCodeExporter]

But that will download whole data if Dataset is remote...

I guess I expected too much ;)

Original comment by klo...@gmail.com on 22 Jun 2013 at 3:20

[GoogleCodeExporter]

I meant, I was hoping that protocol (opendap for example) allows such queries 
that can be mapped to numpy array slicing properties, but wasn't sure

Original comment by klo...@gmail.com on 22 Jun 2013 at 3:23

[GoogleCodeExporter]

You typically will only use coordinate variables for this, and they are 1-d 
arrays (so the overhead of reading the data is not too large).  There's no way 
I know of to avoid reading the data to perform these kinds of comparison 
operations.

Original comment by whitaker.jeffrey@gmail.com on 22 Jun 2013 at 4:58

[GoogleCodeExporter]

Yes, for coordinates it isn't a problem. And maybe there is no interface for 
such binary query exposed anywhere, but I don't know...

I was trying to understand nco operators, if maybe they allow such 
manipulation, but their manual is very confusing to me, right now

Thanks Jeff

Original comment by klo...@gmail.com on 22 Jun 2013 at 6:12

[GoogleCodeExporter]

Original comment by whitaker.jeffrey@gmail.com on 26 Feb 2014 at 2:04

• Changed state: Fixed