pythonprofilers / memory_profiler

Monitor Memory usage of Python code
http://pypi.python.org/pypi/memory_profiler
Other
4.39k stars 380 forks source link

Feature Request: Issue Markers Delimiting Code Regions in .dat #310

Closed william-silversmith closed 3 years ago

william-silversmith commented 3 years ago

When profiling a more complex setup, it can be difficult to determine which part of the code is contributing to the memory usage. One trick that I sometimes use is use time.sleep for an extended period so I can see some kind of regional delimiter between different parts of the code.

However, it would be very handy if one could write something like this:

import memory_profiler

setup_experiment()

memory_profiler.start_region("forloop")
for i in range(100000000):
   # something expensive
memory_profiler.end_region("forloop")

This would inject tokens indicating the start and end of the code region into the .dat which could then be used to automatically shade the plot like I have done manually here:

image
peterjc commented 3 years ago

Your idea would work equally well for my needs - I was looking for a light weight alternative to the @profile decorator, something like @profile(backend=False) or just @label which would just capture the start/end time for that method and use it to label the final graph - but without the overhead of looking at the extra details within that method.

william-silversmith commented 3 years ago

I figured out that function timestamping does something very similar to this idea and is documented in the readme. It suits my purposes.

peterjc commented 3 years ago

@william-silversmith could you expand on that comment slightly please? I don't see any mention of timestamping within https://github.com/pythonprofilers/memory_profiler/blob/master/README.rst

Are you just adding some well placed @profile decorators to your functions/methods?

william-silversmith commented 3 years ago

Yes that's what I ended up doing

On Tue, Nov 23, 2021, 6:40 AM Peter Cock @.***> wrote:

@william-silversmith https://github.com/william-silversmith could you expand on that comment slightly please? I don't see any mention of timestamping within https://github.com/pythonprofilers/memory_profiler/blob/master/README.rst

Are you just adding some well placed @profile decorators to your functions/methods?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pythonprofilers/memory_profiler/issues/310#issuecomment-976431084, or unsubscribe https://github.com/notifications/unsubscribe-auth/AATGQSJM6DSRHK6RODW2FP3UNN4S3ANCNFSM42TLMRFQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.