benfred / py-spy

Sampling profiler for Python programs
MIT License
12.14k stars 400 forks source link

Possible way to add MPI support? #586

Open Chronum94 opened 1 year ago

Chronum94 commented 1 year ago

I had an issue a few years ago about MPI support, unfortunately I did not have a meaningful solution at the time.

Perhaps we can do something along the lines of:

mpirun -np N py-spy record --mpi python script.py

And the --mpi flag triggers a rank-unique filename for the outputs?

Currently I can spawn as many mpi ranks worth of py-spy, but they all attempt to write results to the same file, and of course this doesn't work.

Perhaps we can have rsmpi as an optional dependency that's imported upon detection of the --mpi flag, and then we can have process-unique file writing?

I'm not familiar with rust but I hope this is a somewhat actionable way forward and I haven't said anything obviously stupid.