search

gabrielasd / pyeom

GNU General Public License v3.0
2 stars 0 forks source link

Memory management #52

Closed gabrielasd closed 3 years ago

gabrielasd commented 3 years ago

I didn't get to include memory management before the project's final deadline (raising a MemoryError), but this is something I want to look more into (https://realpython.com/python-memory-management/, https://airbrake.io/blog/python/memoryerror). Also, the current implementation loads completely the integrals and RDM matrices in memory, but this could be done differently, using for example numpy.memmap (https://numpy.org/doc/stable/reference/generated/numpy.memmap.html)

Originally posted by @gabrielasd in https://github.com/gabrielasd/eomee/issues/40#issuecomment-745023656

msricher commented 3 years ago

@gabrielasd as far as I know all of the Python functions we use handle MemoryError gracefully, including the std library and numpy:

>>> a = np.ones((1000000, 1000000, 1000000))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.9/site-packages/numpy/core/numeric.py", line 203, in ones
    a = empty(shape, dtype, order)
numpy.core._exceptions.MemoryError: Unable to allocate 6.94 EiB for an array with shape (1000000, 1000000, 1000000) and data type float64