MemoryError with NetCDF

[GoogleCodeExporter]

1. an ipython session is invoked with qtconsole --pylab 
2. I load a large NetCDF grid (Grid file format: nf (# 18) GMT netCDF format 
(float)  (COARDS-compliant) [DEFAULT]), approx. 1.15 GB
3. I then try to plot with imshow the data

added below are the lines leading up to the error and the error itself.

This is running on OS X 10.7.4 with a recently installed EPD 7.3.

{code}
from scipy.io import netcdf_file as netcdf
data = netcdf('srtm_43_44_05_06_07_08.grd','r').variables['z'][::-1]

fig, ax = subplots()

data.shape
Out[5]: (24004, 12002)

im = ax.imshow(data, aspect=((data.shape[1])/float(data.shape[0])), 
interpolation='none')
---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
<ipython-input-6-f92e4c4c63b5> in <module>()
----> 1 im = ax.imshow(data, aspect=((data.shape[1])/float(data.shape[0])), 
interpolation='none')

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/ma
tplotlib/axes.py in imshow(self, X, cmap, norm, aspect, interpolation, alpha, 
vmin, vmax, origin, extent, shape, filternorm, filterrad, imlim, resample, url, 
**kwargs)
   6743             im.set_clim(vmin, vmax)
   6744         else:
-> 6745             im.autoscale_None()
   6746         im.set_url(url)
   6747 

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/ma
tplotlib/cm.py in autoscale_None(self)
    281         if self._A is None:
    282             raise TypeError('You must first set_array for mappable')
--> 283         self.norm.autoscale_None(self._A)
    284         self.changed()
    285 

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/ma
tplotlib/colors.py in autoscale_None(self, A)
    889         ' autoscale only None-valued vmin or vmax'
    890         if self.vmin is None:
--> 891             self.vmin = ma.min(A)
    892         if self.vmax is None:
    893             self.vmax = ma.max(A)

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/nu
mpy/ma/core.pyc in min(obj, axis, out, fill_value)
   5873 def min(obj, axis=None, out=None, fill_value=None):
   5874     try:
-> 5875         return obj.min(axis=axis, fill_value=fill_value, out=out)
   5876     except (AttributeError, TypeError):
   5877         # If obj doesn't have a max method,

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/nu
mpy/ma/core.pyc in min(self, axis, out, fill_value)
   5054         # No explicit output
   5055         if out is None:
-> 5056             result = self.filled(fill_value).min(axis=axis, 
out=out).view(type(self))
   5057             if result.ndim:
   5058                 # Set the mask

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/nu
mpy/ma/core.pyc in filled(self, fill_value)
   3388             return self._data
   3389         else:
-> 3390             result = self._data.copy()
   3391             try:
   3392                 np.putmask(result, m, fill_value)

MemoryError: 
{/code}

Original issue reported on code.google.com by shahar.k...@gmail.com on 7 Sep 2012 at 8:32

[GoogleCodeExporter]

Not a netcdf4-python issue - you are running out of memory because of the extra 
overhead in matplotlib when trying to display the array.  matplotlib is making 
a copy of the array.  

Original comment by whitaker.jeffrey@gmail.com on 7 Sep 2012 at 12:17

[GoogleCodeExporter]

Original comment by whitaker.jeffrey@gmail.com on 26 Feb 2014 at 2:04

• Changed state: Fixed