gaogaotiantian / viztracer

VizTracer is a low-overhead logging/debugging/profiling tool that can trace and visualize your python code execution.
https://viztracer.readthedocs.io/
Apache License 2.0
5.09k stars 378 forks source link

Ignore python library packages #468

Closed mayurk14 closed 3 weeks ago

mayurk14 commented 2 months ago

Hello,

how to ignore python library packages from tracer, i don't want to know in depth how many python library package has been called and how much time it has consume. i want to check mainly how many times my functions has been spent to execute the code which i have written.

Is there way to achieve this using this library?

one more thing i want to do it, i want to execute my function 100 times and then after i want to compare the result. i know there is a function for merge files, but 100 argument will not make any sense.

Any help will be really helpful

gaogaotiantian commented 2 months ago

Your question is actually more complicated than you thought. We need to break this down to a few pieces.

First of all - what do you mean when you say "ignore Python library packages"? Because ignoring them won't help with estimating the cost of your own code - they just do not show in the trace anymore. The key data viztracer logs is the timestamp, when a function enters and exits. If your function is called at 3s and finishes at 10s, no matter whether you "ignore" the Python library (let's say a sum()) in the middle, it will show as a 7s block. The only thing you'll get, is a trace graph with less details.

viztracer can't "subtract" time spent in other functions from a certain function - you'll have to do that by yourself. You can use flamegraph to get a gist.

And that leads to another issue - how would you define "the code I've written"? Say you wrote some code that contains sum(lst), and sum calls into a Python builtin function, does that count as the code you wrote? viztracer gives you timestamps of function calls and exits, that's it, and you can derive your own conclusion based on that.

Comparing result would be simpler - you can put all the reports in a directory and you should be able to do viztracer --combine my_report_dir/*

mayurk14 commented 2 months ago

thanks for the quick reply.

You got it correct. I want to see trace graph with less details. if sum() has taken 7 in the block, that should be fine to not see that detail as i can see def a() has taken 10 seconds. Once i figure out which function has taken more time, lets say def b() has taken 15 seconds, so after figure out this, i want to optimise the code/function which takes more time compare to the spent time on the function which are quicker.

gaogaotiantian commented 2 months ago

There are two options for you to explore - --ignore_c_function and --exclude_files. You can check out the docs for the usage (should be easy to follow). If you want white-list style, check out --log_sparse. Let me know if these can't solve your issue.