radio-astro-tools / spectral-cube

Library for reading and analyzing astrophysical spectral data cubes
http://spectral-cube.rtfd.org
BSD 3-Clause "New" or "Revised" License
95 stars 61 forks source link

Improve docs for big data #60

Open astrofrog opened 10 years ago

astrofrog commented 10 years ago

As mentioned in #15 by @keflavich regarding big_data.rst:

"If we're going to identify a method that is more efficient, we should illustrate how to use the more efficient method. I think this calls for a refactor of the _apply_numpy_function method to use @ChrisBeaumont's slab-by-slab method if axis is passed and add a memmap=False kwarg in the _apply_numpy_function & inherited tools. I guess this ought to go to a separate issue."

ChrisBeaumont commented 10 years ago

I can implement this. However, I don't think I understand what memmap=False is meant to communicate. Does this specify whether the computation happens slabwise? If so, I think we should use the same convention as moment (how='cube'|'slice'|'ray')

ChrisBeaumont commented 10 years ago

Along these lines, we might want to add a quick tutorial discussing what a scientist needs to keep in mind to work productively with large cubes. I see lots of advice scattered through the docs (which is good), but we might want to have the equivalent of "getting started with big cubes" section

ChrisBeaumont commented 10 years ago

78 Adds the suggested tutorial, and partially implements slicewise-reduction for numpy methods.

There is a bit more that could be done in this regard:

astrofrog commented 10 years ago

Most of what is done so far is fine for 0.1, re-scheduling this for 0.2 for the remaining issues.