search

radio-astro-tools / spectral-cube

Library for reading and analyzing astrophysical spectral data cubes
http://spectral-cube.rtfd.org
BSD 3-Clause "New" or "Revised" License
97 stars 65 forks source link

dask apply_numpy_function should allow scipy functions #741

Open keflavich opened 3 years ago

keflavich commented 3 years ago

The check we have for numpy ufunc-ness is a bit clunky; scipy functions are often (generally?) numpy ufuncs, so we should allow them.

https://github.com/radio-astro-tools/spectral-cube/blob/c3a9eceb5ace5cab774fe4fce58a4a40c2b1b024/spectral_cube/dask_spectral_cube.py#L470-L471

keflavich commented 3 years ago

I'm not sure what I said is true; the scipy functions might force loading into memory.

keflavich commented 3 years ago 
>>> from dask_image.ndfilters import median_filter
>>> cubenn = cube.apply_function(median_filter, size=(3,3,3))
Traceback (most recent call last):
  File "<ipython-input-12-6e6d53f69d06>", line 1, in <module>
    cubenn = cube.apply_function(median_filter, size=(3,3,3))
  File "/blue/adamginsburg/adamginsburg/casa/casa-CAS-13609-1/lib/py/lib/python3.6/site-packages/spectral_cube/dask_spectral_cube.py", line 76, in wrapper
    cube = function(self, *args, **kwargs)
  File "/blue/adamginsburg/adamginsburg/casa/casa-CAS-13609-1/lib/py/lib/python3.6/site-packages/spectral_cube/dask_spectral_cube.py", line 115, in wrapper
    out = function(self, *args, **kwargs)
  File "/blue/adamginsburg/adamginsburg/casa/casa-CAS-13609-1/lib/py/lib/python3.6/site-packages/spectral_cube/dask_spectral_cube.py", line 399, in apply_function
    out = function(self.flattened(), **kwargs)
  File "/blue/adamginsburg/adamginsburg/casa/casa-CAS-13609-1/lib/py/lib/python3.6/site-packages/spectral_cube/spectral_cube.py", line 1057, in flattened
    data = self._mask._flattened(data=self._data, wcs=self._wcs, view=slice)
AttributeError: 'NoneType' object has no attribute '_flattened'