Tracing of decorated functions has changed between 7.2.0 and 7.3.0

[gitlab-importer]

In Heptapod by bitbucket_importer on Dec 31, 2019, 16:06

Created originally on Bitbucket by ned (Ned Batchelder)

The line numbers compiled into code objects has changed for decorated functions

In 7.2.0, the def line was included for decorated functions. In 7.3.0, the def line is not included. Decorated methods include the def line in all versions.

CPython has always included the def line for decorated functions. For decorated methods, they changed in 3.8 to include the def line for decorated methods also.

CPython changed their behavior in Python 3.8 to include the def line for methods, but not functions.

Here’s a test case. dis shows the line numbers along the left. 7.3.0 is missing line numbers 5 and 25:

$ cat decorators.py
# 1

@foo(3)
@bar
def func(x, y=5):
    return 6

class Foo:
    '''9'''
    @foo
    def __init__(self):
        '''12'''
        return 13

    @foo(
        16,
        17,
    )
    def meth(self):
        return 20

@foo(
    23
)
def func(x=25):
    return 26

$ /usr/local/pythonz/pythons/PyPy3-v7.2.0/bin/pypy3 -m dis decorators.py
  3           0 LOAD_NAME                0 (foo)
              2 LOAD_CONST               0 (3)
              4 CALL_FUNCTION            1

  4           6 LOAD_NAME                1 (bar)

  5           8 LOAD_CONST               1 (5)
             10 BUILD_TUPLE              1
             12 LOAD_CONST               2 (<code object func at 0x000000010a7b76e8, file "decorators.py", line 3>)
             14 LOAD_CONST               3 ('func')
             16 MAKE_FUNCTION            1
             18 CALL_FUNCTION            1
             20 CALL_FUNCTION            1
             22 STORE_NAME               2 (func)

  8          24 LOAD_BUILD_CLASS
             26 LOAD_CONST               4 (<code object Foo at 0x000000010a7b7970, file "decorators.py", line 8>)
             28 LOAD_CONST               5 ('Foo')
             30 MAKE_FUNCTION            0
             32 LOAD_CONST               5 ('Foo')
             34 CALL_FUNCTION            2
             36 STORE_NAME               3 (Foo)

 22          38 LOAD_NAME                0 (foo)

 23          40 LOAD_CONST               6 (23)
             42 CALL_FUNCTION            1

 25          44 LOAD_CONST               7 (25)
             46 BUILD_TUPLE              1
             48 LOAD_CONST               8 (<code object func at 0x000000010a7b7a48, file "decorators.py", line 22>)
             50 LOAD_CONST               3 ('func')
             52 MAKE_FUNCTION            1
             54 CALL_FUNCTION            1
             56 STORE_NAME               2 (func)
             58 LOAD_CONST               9 (None)
             60 RETURN_VALUE

$ /usr/local/pythonz/pythons/PyPy3-v7.3.0/bin/pypy3 -m dis decorators.py
  3           0 LOAD_NAME                0 (foo)
              2 LOAD_CONST               0 (3)
              4 CALL_FUNCTION            1

  4           6 LOAD_NAME                1 (bar)
              8 LOAD_CONST               1 ((5,))
             10 LOAD_CONST               2 (<code object func at 0x000000010bc93e80, file "decorators.py", line 3>)
             12 LOAD_CONST               3 ('func')
             14 MAKE_FUNCTION            1
             16 CALL_FUNCTION            1
             18 CALL_FUNCTION            1
             20 STORE_NAME               2 (func)

  8          22 LOAD_BUILD_CLASS
             24 LOAD_CONST               4 (<code object Foo at 0x000000010bcd41d0, file "decorators.py", line 8>)
             26 LOAD_CONST               5 ('Foo')
             28 MAKE_FUNCTION            0
             30 LOAD_CONST               5 ('Foo')
             32 CALL_FUNCTION            2
             34 STORE_NAME               3 (Foo)

 22          36 LOAD_NAME                0 (foo)

 23          38 LOAD_CONST               6 (23)
             40 CALL_FUNCTION            1
             42 LOAD_CONST               7 ((25,))
             44 LOAD_CONST               8 (<code object func at 0x000000010bcd42a8, file "decorators.py", line 22>)
             46 LOAD_CONST               3 ('func')
             48 MAKE_FUNCTION            1
             50 CALL_FUNCTION            1
             52 STORE_NAME               2 (func)
             54 LOAD_CONST               9 (None)
             56 RETURN_VALUE

‌

[gitlab-importer]

In Heptapod by @arigo on Jan 1, 2020, 19:33

I think this is an irrelevant detail. The reason for why the line is missing is because the default argument is found to be a constant in pypy3 7.3.0, but not in pypy3 7.2.0. The following two examples give the same result in each case:

#!python

@foo
def f(x):     # this line not included by dis
    pass

#!python

@foo
def f(x=y):        # this line IS included by dis
    pass

But the following code gives a different result:

#!python

@foo
def f(x=25):
    pass

The difference is only that this last piece of code compiles like the first one above in 7.3.0 (because the constant 25 doesn't really need line numbers, as it cannot fail), whereas it used to compile like the 2nd one above in 7.2.0 (because it used to involve a BUILD_TUPLE, which could fail).

I think the difference is only caused by removing pointless, never-executed line numbers.

If I'm wrong, I can be convinced with another example.

[gitlab-importer]

In Heptapod by bitbucket_importer on Jan 6, 2020, 12:48

Created originally on Bitbucket by ned (Ned Batchelder)

Thanks for the explanation. I follow these details because coverage.py needs to understand what lines could be traced. I write these bugs to be sure the language devs know the change has happened.

[gitlab-importer]

In Heptapod by @arigo on Jan 6, 2020, 17:52

Found out how CPython 3.6.9 does it and copied it exactly in d36692105171: it emits the lineno of the last item of the tuple when the tuple of defaults is a constant.