pyutils / line_profiler

Line-by-line profiling for Python
Other
2.77k stars 120 forks source link

Profiling inner functions #300

Open 6nc0r6-1mp6r0 opened 1 month ago

6nc0r6-1mp6r0 commented 1 month ago

In my code, I have numerous inner functions which, for various reasons, I would like to leave as inner functions if at all possible. However, line_profiler does not appear to profile these functions by default. For example, the following code

from line_profiler import LineProfiler
import numpy as np

m = 2
def main():
    n = 3
    def called_func(x):
        y = x**m
        z = x + y + n
        return z
    for i in range(10):
        x = np.random.rand()
        z = called_func(x)
    print(z)

lp = LineProfiler()
lp_wrapper = lp(main)
lp_wrapper()
lp.print_stats()

returns

4.335277136950235
Timer unit: 1e-07 s

Total time: 0.0007159 s
File: c:\Users\...\AppData\Local\Programs\Python\Python312\...\line_profiler_test.py
Function: main at line 5

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     5                                           def main():
     6         1         54.0     54.0      0.8      n = 3
     7         1         10.0     10.0      0.1      def called_func(x):
     8                                                   y = x**m
     9                                                   z = x + y + n
    10                                                   return z
    11        11         83.0      7.5      1.2      for i in range(10):
    12        10        218.0     21.8      3.0          x = np.random.rand()
    13        10        263.0     26.3      3.7          z = called_func(x)
    14         1       6531.0   6531.0     91.2      print(z)

Is it possible to modify the code above so that called_func is also profiled in a second block of output (while leaving it as an inner function)? A command-line approach would be fine if that would be better in some respect than the above in-code approach.

Several notes. I am working on Windows 10. So far, I have made several attempts to include a line like lp.add_function(main.called_func) and have tried to incorporate ideas from this Medium article and this Stack Overflow post without success. I also saw the comment below on the main GitHub page, but could not figure out how to implement it.

Nesting is common when using LineProfiler as a decorator. In order to support nesting, use enable_by_count() and disable_by_count(). These functions will increment and decrement a counter and only actually enable or disable the profiler when the count transitions from or to 0.

Any help would be appreciated.

ta946 commented 1 month ago

you have 2 options that require you to modify the code, and 1 that doesnt:

  1. you can do what you were doing before, just create new wrapped function called_func_wrapped = lp(called_func) and use that insead z = called_func_wrapped (x)

  2. you can use decorators instead of returning a wrapped function

from line_profiler import LineProfiler
import numpy as np

lp = LineProfiler()

m = 2
@lp
def main():
    n = 3
    @lp
    def called_func(x):
        y = x**m
        z = x + y + n
        return z
    for i in range(10):
        x = np.random.rand()
        z = called_func(x)
    print(z)

main()
lp.print_stats()
  1. you can use auto profiling so theres no need edit the code, but you will have to change the way you run the script and you will only get the output after the script has finished running instead of getting it in the script like you are now. if that is acceptable https://kernprof.readthedocs.io/en/latest/line_profiler.autoprofile.html#module-line_profiler.autoprofile