python / cpython

The Python programming language
https://www.python.org
Other
63.86k stars 30.56k forks source link

LOAD_FAST access names outside of co_varnames #122794

Open MatthieuDartiailh opened 3 months ago

MatthieuDartiailh commented 3 months ago

Documentation

LOAD_FAST is documented (https://docs.python.org/3.13/library/dis.html#opcode-LOAD_FAST) as: Pushes a reference to the local co_varnames[var_num] onto the stack.

However running dis on the following snippet shows that it may load other names:


def simple_cellvar():  # a cellvar in f
    a = 1

    def g():  # a freevar in g
        return a

    return g

import dis

dis.dis(simple_cellvar)
print()
print(f"Var names {simple_cellvar.__code__.co_varnames}")

Output

--           MAKE_CELL                1 (a)

   5           RESUME                   0

   6           LOAD_CONST               1 (1)
               STORE_DEREF              1 (a)

   8           LOAD_FAST                1 (a)      <---- a does not appear in the co_varnames !!!
               BUILD_TUPLE              1
               LOAD_CONST               2 (<code object g at 0x0000027404FB0ED0, ...)
               MAKE_FUNCTION
               SET_FUNCTION_ATTRIBUTE   8 (closure)
               STORE_FAST               0 (g)

  11           LOAD_FAST                0 (g)
               RETURN_VALUE

Disassembly of <code object g at 0x0000027404FB0ED0, ...>:
  --           COPY_FREE_VARS           1

   8           RESUME                   0

   9           LOAD_DEREF               0 (a)
               RETURN_VALUE

Var names ('g',)
gaogaotiantian commented 3 months ago

Right, now that we effectly store all of them in the "fast locals", we should clean up the docs about it. However, I'm not sure what's the best way to proceed. Should we create a new term for the local names? Or should we change the definition of varnames?

MatthieuDartiailh commented 3 months ago

I asked this question because I maintain https://github.com/MatthieuDartiailh/bytecode and I usually stumble about undocumented bytecode changes each time I start the work to support the latest Python.

I do not think the definition of varnames should be changed since cell values never appeared there but I may be missing something. Currently the len of varnames matches the value passed to types.CodeType nlocals parameters. Does this remain true or has it also changed ?

kamilturek commented 3 months ago

I agree that the definition of varnames should stay.

I see that some opcodes in dis.rst already refer to "fast locals".

Examples:

Could a similar thing be done for LOAD_FAST, LOAD_FAST_FAST, and related - write they access "fast locals" instead of co_varnames? That would be consistent.

MatthieuDartiailh commented 3 months ago

If the notion of fast locals is clearly defined I am fine with it.

While considering this, I had a second though: since load fast may access a cell, should it have the has_free flag set ? I do not think it should, but I am finding more and more difficult to understand what the dis.hasfree list covers if cell vars may be accessed by opcodes outside of it.