search

pythonprofilers / memory_profiler

Monitor Memory usage of Python code
http://pypi.python.org/pypi/memory_profiler
Other
4.39k stars 380 forks source link

[BUG] Reused memory from previous allocations is not reported #348

Open NightMachinery opened 2 years ago

NightMachinery commented 2 years ago 
!pip install -U memory_profiler
%load_ext memory_profiler

for i in range(4):
  n = 10**(5+4)
  %memit np.ones((n,), dtype=np.float32)

None
peak memory: 4018.02 MiB, increment: 3818.63 MiB
peak memory: 4018.07 MiB, increment: 0.00 MiB
peak memory: 4018.07 MiB, increment: 0.00 MiB
peak memory: 4018.07 MiB, increment: 0.00 MiB

As you see, while the memory consumption of the line np.ones((n,), dtype=np.float32) is constant, only the first measurement is correct; the next measurements seem to not notice that the Python interpreter is reusing memory it had previously claimed from the OS and freed in garbage collection.

Is there any way to handle this problem? It makes it difficult to find where memory is being actually used, as opposed to finding the very first big allocation.