search

gems-uff / noworkflow

Supporting infrastructure to run scientific experiments without a scientific workflow management system.
http://gems-uff.github.io/noworkflow
MIT License
120 stars 28 forks source link

How to Profile noworkflow #122

Closed jiwonjoung closed 5 months ago

jiwonjoung commented 6 years ago

Hello all -- I am interested in profiling noworkflow using a Python profiler like cProfile but there seems to be a name collision between some functions in cProfile and those in the noworkflow profiler. This breaks the profiler on noworkflow. I am interested in seeing which parts of noworkflow are consuming the most time.

Any tips on how to do this?

JoaoFelipe commented 6 years ago

When we tried to profile noWorkflow, we faced the same issue =/ It was a while ago, and I don't quite remember what happened, but I think it was not a name collision issue. It was related to to using two profilers in Python (noWorkflow uses a profiler to collect execution provenance).

We couldn't fix it at the time and we implemented the following decorator to collect and save the durations: https://github.com/gems-uff/noworkflow/blob/master/capture/noworkflow/now/utils/metaprofiler.py

The new version of noWorkflow (branch 2.0-alpha) does not use a profiler to collect provenance, but it is still in development and may not work for many scripts

JoaoFelipe commented 5 months ago

Closing the issue since it is related to noWorklow 1.0