memory_usage calls the target function multiple times if interval is too long

[antoche]

Hi,

According to the docs, memory_usage is supposed to call the target function just once. But depending on the arguments, it sometimes calls the target function more than once. Example:

>>> import numpy as np
>>> import memory_profiler
>>> def go():
...     print(np.random.rand(1))
...     a = np.ones((1000, 1000, 1000))
...     print(a.shape)
... 
>>> memory_profiler.memory_usage(go, interval=.1, max_usage=True, retval=True)
[ 0.18843382]
(1000, 1000, 1000)
(7539.58203125, None)
>>> memory_profiler.memory_usage(go, interval=3., max_usage=True, retval=True)
[ 0.11062882]
(1000, 1000, 1000)
[ 0.95484454]
(1000, 1000, 1000)
(7097.71484375, None)

This was tested using python-3.7.3, memory_profiler-0.57.0 and numpy-1.19.5.

This behaviour is dangerous if either the caller or target function were expecting a single call to be performed. Either the docs should mention that the target function may be called an arbitrary number of times, or the logic should be fixed to ensure it's always only called a single time.

Thanks, A.

[fabianp]

Thanks Antoine,

could you please send a pull request to change the docs? Thanks!

On Mon, Jul 26, 2021 at 5:51 PM Antoine Bouthors @.***> wrote:

Hi,

According to the docs, memory_usage is supposed to call the target function just once. But depending on the arguments, it sometimes calls the target function more than once. Example:

import numpy as np import memory_profiler def go(): ... print(np.random.rand(1)) ... a = np.ones((1000, 1000, 1000)) ... print(a.shape) ... memory_profiler.memory_usage(go, interval=.1, max_usage=True, retval=True) [ 0.18843382] (1000, 1000, 1000) (7539.58203125, None) memory_profiler.memory_usage(go, interval=3., max_usage=True, retval=True) [ 0.11062882] (1000, 1000, 1000) [ 0.95484454] (1000, 1000, 1000) (7097.71484375, None)

This was tested using python-3.7.3, memory_profiler-0.57.0 and numpy-1.19.5.

This behaviour is dangerous if either the caller or target function were expecting a single call to be performed. Either the docs should mention that the target function may be called an arbitrary number of times, or the logic should be fixed to ensure it's always only called a single time.

Thanks, A.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pythonprofilers/memory_profiler/issues/322, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACDZB7SEPRF3Q3IPN7MFB3TZXKE7ANCNFSM5BA6QVIA .

[antoche]

I would have no idea what the docs should say, sorry. Personally I think it's the behaviour that's incorrect. As a user, if I'm adding instrumentation around a call site, I'm expecting the call to still only occur once and exactly once. The fact that the instrumentation library may arbitrarily decide to call my function multiple times is dangerous and a deal-breaker IMO. And I can't think of a reason why it should need to call it multiple times.