search

radio-astro-tools / spectral-cube

Library for reading and analyzing astrophysical spectral data cubes
http://spectral-cube.rtfd.org
BSD 3-Clause "New" or "Revised" License
95 stars 61 forks source link

Error message when doing huge operation doesn't give a good suggestion for making operation less huge #915

Open abulatek opened 2 weeks ago

abulatek commented 2 weeks ago

When doing cube.median(axis=0), I get the following error message:

ValueError: This function (<function BaseSpectralCube._apply_everywhere at 0x14f675092790>) requires loading the entire cube into memory, and the cube is large (503316480 pixels), so by default we disable this operation. To enable the operation, set `cube.allow_huge_operations=True` and try again.  Alternatively, you may want to consider using an approach that does not load the whole cube into memory by specifying how='slice' or how='ray'.  See https://spectral-cube.readthedocs.io/en/latest/big_data.html for details.

The suggestion to use how='ray' gives the same error message, as does using iterate_rays=True (which is not suggested by the error message). Adam said this is due to some missing logic cases.

SpacialTree commented 1 week ago

Hey Alyssa!

What version of spectral-cube are you using? The most recent dev version?

abulatek commented 1 week ago

The last version I pulled was this one, which was designed to fix this issue: https://github.com/radio-astro-tools/spectral-cube/pull/916/files. It looks like it probably does fix this issue, but I was using dask spectral cubes, which don't take these keywords anyway.