`frame.f_lineno` can be none

[sternj]

frame.f_lineno can be None in limited circumstances in Python 3.10 as documented here. This can cause an issue in the Statistical profiler, as the line max(self.line_dict) can error out in this case here.

[vpelletier]

Thanks for this report.

I have a one-liner tentative fix, but I cannot get the issue to occur without it to begin with, neither with the code from https://github.com/python/cpython/issues/94485 nor by executing in an interpreter a few of the test changes from https://github.com/python/cpython/commit/1bfe83a114da3939c00746fc44dc5da7f56f525f .

So I cannot verify if my tentative fix would actually work.

The Python/compile.c code changed enough between 3.10 and 3.11 that I am not 100% sure the backport did not happen, and I have not even looked at patches my distro could have applied on top of it. For reference, I am testing with Debian sid python3.10 3.10.8-3 .

Do you have some code reproducing this issue ?

Alternatively, could you apply this patch and check if it is enough for the failure to disappear ?

diff --git a/pprofile/__init__.py b/pprofile/__init__.py
index 29a6ad6..a3fefcd 100755
--- a/pprofile/__init__.py
+++ b/pprofile/__init__.py
@@ -1026,7 +1026,11 @@ class Profile(ProfileBase, ProfileRunnerBase):
         local_trace = self._local_trace
         if local_trace is not None:
             event_time = time()
-            callee_entry = [event_time, 0, frame.f_lineno, event_time, 0]
+            # Workaround f_lineno being None in some edge cases in some python
+            # 3.10 versions. Apparently this only affects module imports, which
+            # are traced as calls:
+            # https://github.com/python/cpython/issues/94485
+            callee_entry = [event_time, 0, frame.f_lineno or 0, event_time, 0]
             try:
                 stack, callee_dict = self.stack
             except TypeError:

[P403n1x87]

I can repro this consistently on OSX

pip install py-sudoku

❯ cat main.py
from sudoku import Sudoku
from time import monotonic as time

end = time() + 5

while time() <= end:
    Sudoku(3).solve()

❯ pprofile -s 0.01 -m main
Command line: main
Total duration: 5.00392s
File: .venv/lib/python3.10/site-packages/sudoku/sudoku.py
File duration: 0s (0.00%)
Line #|      Hits|         Time| Time per hit|      %|Source code
------+----------+-------------+-------------+-------+-----------
Traceback (most recent call last):
  File "/Users/gabriele.tornetta/p403n1x87/sudoku/.venv/bin/pprofile", line 33, in <module>
    sys.exit(load_entry_point('pprofile==2.1.0', 'console_scripts', 'pprofile')())
  File "/Users/gabriele.tornetta/p403n1x87/sudoku/.venv/lib/python3.10/site-packages/pprofile/__init__.py", line 1492, in main
    getattr(prof, format_dict[options.format])(
  File "/Users/gabriele.tornetta/p403n1x87/sudoku/.venv/lib/python3.10/site-packages/pprofile/__init__.py", line 780, in annotate
    last_line = file_timing.getLastLine()
  File "/Users/gabriele.tornetta/p403n1x87/sudoku/.venv/lib/python3.10/site-packages/pprofile/__init__.py", line 256, in getLastLine
    max(self.line_dict) if self.line_dict else 0,
TypeError: '>' not supported between instances of 'NoneType' and 'int'

[micsthepick]

Replicates on Termux:

pprofile --include dp.py -s 0.1 dp.py
... dp.py output elided...
Command line: dp.py
Total duration: 2.63185s
File: dp.py
File duration: 0s (0.00%)
Line #|      Hits|         Time| Time per hit|      %|Source code
------+----------+-------------+-------------+-------+-----------
Traceback (most recent call last):
  File "/data/data/com.termux/files/usr/bin/pprofile", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pprofile/__init__.py", line 1492, in main
    getattr(prof, format_dict[options.format])(
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pprofile/__init__.py", line 780, in annotate
    last_line = file_timing.getLastLine()
                ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pprofile/__init__.py", line 256, in getLastLine
    max(self.line_dict) if self.line_dict else 0,
    ^^^^^^^^^^^^^^^^^^^
TypeError: '>' not supported between instances of 'int' and 'NoneType'

what works for me:

max(x if x is not None else 0 for x in self.line_dict) if self.line_dict else 0,

[vpelletier]

Thanks for these recent reports and for confirming code which avoids this failure.

I pushed an extended version of the patch I proposed for testing above (to avoid the bad values finding their way in the data structure, rather than fixing them later... I think this should be cheap enough): 89dd31f14cb92f8d4d02c4ef332f640bb164ca64 .

Please let me know if this fixes the issue.

[vpelletier]

This tentative fix is now released in 2.2.0 .