benfred / py-spy

Sampling profiler for Python programs
MIT License
12.16k stars 401 forks source link

How to profile only certain part of Python process? #531

Open EricCousineau-TRI opened 1 year ago

EricCousineau-TRI commented 1 year ago

Not sure if this has already been asked, but was curious if there's a best way to only profile part of process - I'd also be willing to make code modifications if need be.

Motivation is two-fold:

I found this works 'aight, albeit somewhat sloppy:

from contextlib import contextmanager
import os
import signal
import subprocess

@contextmanager
def use_py_spy(output_file, *, sudo=False):
    """Use py-spy in specific context."""
    args = ["py-spy", "record", "-o", output_file, "--pid", str(os.getpid())]
    if sudo:
        args = ["sudo"] + args
    p = subprocess.Popen(args)
    # TODO(eric.cousineau): Startup time of py-spy may lag behind other
    # instructions. However, maybe can assume this isn't critical for profiling
    # purposes?
    yield
    p.send_signal(signal.SIGINT)
    p.wait()

def main():
    with use_py_spy("/tmp/crazy_stuff.svg"):
       do_crazy_stuff()

...

See usecase for CPython API extension stuff:

Thanks for the awesome tool!

olejorgenb commented 1 year ago

Maybe https://github.com/joerick/pyinstrument is more suited for this? It is designed to be called from within python.