dfeneyrou / palanteer

Visual Python and C++ nanosecond profiler, logger, tests enabler
Other
2.1k stars 88 forks source link

Python multiprocessing support #36

Closed danielhrisca closed 2 years ago

danielhrisca commented 2 years ago

Hello,

I've traced a Python application that uses multiprocessing. The way the processes are started is like this:

I can only see the main process trace information.

Is there something that I'm missing?

dfeneyrou commented 2 years ago

Sorry for late answer... To enable profiling, the minimum actions that shall occur in any python program/process is to import the module palanteer and call the function plInitAndStart (among other things, this installs the hooks required to track the functions calls).

If your program creates other processes (with calls to subprocess or system calls), the enabled state of the profiling is not automatically propagated to them because they are independent executables. They even can be made with any language, be a shell script, a binary, ...

So propagating the profiling enabled state to child processes cannot be done (IMHO) automatically and requires some dedicated code in each parent python program. For instance, if palanteer._is_activated is True (an additional getter in the API is indeed needed...):

dfeneyrou commented 2 years ago

Please reopen if you need more information.