python / cpython

The Python programming language
https://www.python.org
Other
63.37k stars 30.33k forks source link

FunctionType.__new__ can generate functions that immediately crash #78373

Closed df79943f-4aee-4531-a00d-c6b12816eb70 closed 3 years ago

df79943f-4aee-4531-a00d-c6b12816eb70 commented 6 years ago
BPO 34192
Nosy @rhettinger, @mr-nfamous, @tirkarthi, @sweeneyde

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields: ```python assignee = None closed_at = created_at = labels = ['interpreter-core', '3.7', 'type-crash'] title = 'FunctionType.__new__ can generate functions that immediately crash' updated_at = user = 'https://github.com/mr-nfamous' ``` bugs.python.org fields: ```python activity = actor = 'Dennis Sweeney' assignee = 'none' closed = True closed_date = closer = 'Dennis Sweeney' components = ['Interpreter Core'] creation = creator = 'bup' dependencies = [] files = [] hgrepos = [] issue_num = 34192 keywords = [] message_count = 3.0 messages = ['322174', '322338', '404220'] nosy_count = 4.0 nosy_names = ['rhettinger', 'bup', 'xtreak', 'Dennis Sweeney'] pr_nums = [] priority = 'normal' resolution = 'wont fix' stage = 'resolved' status = 'closed' superseder = None type = 'crash' url = 'https://bugs.python.org/issue34192' versions = ['Python 3.4', 'Python 3.5', 'Python 3.6', 'Python 3.7'] ```

df79943f-4aee-4531-a00d-c6b12816eb70 commented 6 years ago

The following program crashes on 3.3.1, 3.4.2, 3.5.2, and 3.6.1 because despite having a non-empty co_cellvars slot due to the generator object, the NOFREE flag was set. 3.7 isn't immune to some bad behavior here, either.

While I only have access to 3.7.03b right now, I noted the assertion at the bottom failing because it appears CodeType.__new__ is silently stripping the NOFREE flag out, which is why it's highlighted as well.

Since the interpreter doesn't actually use FunctionType.__new__ it shouldn't hurt performance too much to add a check for when NOFREE is set that co_cellvars and co_freevars are indeed empty. Anyway, the code:

from functools import reduce
from itertools import repeat
from operator import or_
from types import FunctionType, CodeType

OPTIMIZED, NEWLOCALS, NOFREE = 1, 2, 64 FLAGS = [OPTIMIZED, NEWLOCALS, NOFREE] fields=('argcount kwonlyargcount nlocals stacksize flags code consts ' 'names varnames filename name firstlineno lnotab freevars cellvars').split()

def edit_code(code, *args, **kws):
    """Construct a new code object using `code` as a template"""

    params = []
    atrs = ('co_%s'%a for a in fields)
    kwds = map(kws.pop, fields, repeat(kws))
    for arg, kwv, k in zip(args, kwds, atrs):
        if kwv is not kws:
            raise TypeError("edit_code() got multiple parameters for %r"%k)
        params.append(arg)
    for kwv, atr in zip(kwds, atrs):
        params.append(kwv if kwv is not kws else getattr(code, atr))
    if kws:
        k, v = kws.popitem()
        raise TypeError("edit_code() got unexpected keyword argument %r"%k)
    return CodeType(*params)

def get_co_flags(flags):
    if isinstance(flags, FunctionType):
        flags = flags.__code__.co_flags
    elif isinstance(flags, CodeType):
        flags = flags.co_flags
    return reduce(or_, (i for i in FLAGS if flags & i))

if __name__ == '__main__':
    co = get_co_flags.__code__
    ns = get_co_flags.__globals__
    flags = co.co_flags
    assert flags == OPTIMIZED|NEWLOCALS
    assert NOFREE == 64
    a = FunctionType(edit_code(co, flags=flags), ns)
    b = FunctionType(edit_code(co, flags=flags|NOFREE), ns)
    # this assertion fails on 3.7.0b3
    assert b.__code__.co_flags == OPTIMIZED|NEWLOCALS|NOFREE
    print('calling a...')
    a(get_co_flags)
    t = input("Blow up the universe? y/n : ")
    if t != 'n':
        b(get_co_flags)
tirkarthi commented 6 years ago

The assertion also fails on master. I did some manual git blame work. The assertion passes with commit 7324b5ce8e7c031a0a3832a6a8d7c639111ae0ff. It fails with the next commit 078f1814f1a4413a2a0fdb8cf4490ee0fc98ef34 (https://bugs.python.org/issue32176) that has some changes with respect to CO_NOFREE. The change was released with 3.7.0a3 as noted in the comment (https://bugs.python.org/issue32176#msg307512) and hence the assertion should fail on 3.7.0b3 also.

# commit 7324b5c

➜ cpython git:(7324b5c) ✗ ./python Python 3.7.0a2+ (v3.7.0a2-334-g7324b5c:7324b5c, Jul 25 2018, 06:34:52) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information.

>> ➜ cpython git:(7324b5c) ✗ ./python bpo-34192.py calling a... Blow up the universe? y/n : n

# commit 078f181

➜  cpython git:(078f181) ✗ ./python
Python 3.7.0a2+ (v3.7.0a2-335-g078f181:078f181, Jul 25 2018, 06:36:32)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
➜  cpython git:(078f181) ✗ ./python bpo34192.py
Traceback (most recent call last):
  File "bpo34192.py", line 44, in <module>
    assert b.__code__.co_flags == OPTIMIZED|NEWLOCALS|NOFREE
AssertionError

Thanks

sweeneyde commented 3 years ago

From https://github.com/python/cpython/blob/main/Lib/test/crashers/bogus_code_obj.py :

""" Broken bytecode objects can easily crash the interpreter.

This is not going to be fixed. It is generally agreed that there is no point in writing a bytecode verifier and putting it in CPython just for this. Moreover, a verifier is bound to accept only a subset of all safe bytecodes, so it could lead to unnecessary breakage. """

Since this is messing with implementation details of code objects, I'll close this as "won't fix".