search

nvdv / vprof

Visual profiler for Python
BSD 2-Clause "Simplified" License
3.95k stars 154 forks source link

How to profile long running application ? #92

Open eromoe opened 5 years ago

eromoe commented 5 years ago

Hello,

I have a project need multiprocessing and would take over 4 hour in a 36cores vps . My local machine is windows, has some problem to run mprof .

So I run vprof -c cmh "run.py salecount -n 1" --output-file profile.json on a remote vps . Because vprof didn't support multiprocessing , I can't wait it done . Abort from execution, it didn't export any files .

What is the correct way in my case ??

nvdv commented 5 years ago

I would suggest running your program on smaller input if it's possible.

TeamSpen210 commented 5 years ago

You don't need to run your full application. Comment out code and tweak the main loop so it runs the program directly instead of multiprocessing, and runs on say a quarter of the data you're using. You're looking for the functions taking high proportions of time, not the exact full runtime.