tskisner
commented
8 years ago I now have a proof of principle code in a separate branch that implements aligned memory allocation in pure C. This is the start of the core C library that we need in any case for lots of things. I can use cython to create Numpy arrays from scratch with these memory buffers, and also disable the destruction of that memory when the numpy array is destroyed. These memory buffers are allocated outside of the python memory pool, so can be returned to the OS. Based on memory_profiler output, this technique seems to work. However, it does require care to ensure that all numpy arrays with references to that buffer are out of use before deallocation.
Currently the read/write methods of the TOD base class are being abused to implement timestream caching. This prevents descendant classes from using this cache. This should be moved into an explicit, separate object. The TOD base class can still contain an instance of this "cache" object, but this will then be available to all descendents. It also opens the possibility for us to explicitly manage the memory of this cache outside of python, which may be needed to address things like issue #13.