python-trio / trio

Trio – a friendly Python library for async concurrency and I/O
https://trio.readthedocs.io
Other
6.22k stars 342 forks source link

KI protection f_locals materialization results in reference cycles on 3.12 and below #3108

Closed graingert closed 1 month ago

graingert commented 1 month ago

consider:

import trio
import gc

class MyException(Exception):
    pass

async def demo():
    async def handle_error():
        try:
            raise MyException
        except MyException as e:
            exceptions.append(e)

    exceptions = []
    try:
        async with trio.open_nursery() as n:
            n.start_soon(handle_error)
        raise ExceptionGroup("errors", exceptions)
    finally:
        del exceptions

async def main():
    exc = None
    try:
        await demo()
    except* MyException as excs:
        exc = excs.exceptions[0]

    assert exc is not None
    print(gc.get_referrers(exc))
    exc.__traceback__.tb_frame.f_locals  # re-materialize f_locals to sync any deletions
    print(gc.get_referrers(exc))

trio.run(main)

this prints:

[[MyException()]]
[]

so there's a reference cycle from the exc.__traceback__.tb_frame.f_locals["exceptions"][0] is exc, which gets cleared when you re-materialize f_locals

this is caused by this materialization of the frame f_locals https://github.com/python-trio/trio/blob/2a66a0d149ec0796a542fcf0be726e7b81aba301/src/trio/_core/_run.py#L1874

See also https://github.com/agronholm/anyio/pull/809

graingert commented 1 month ago

I can work around this by passing ExceptionGroup a copy() of exceptions, and calling exceptions.clear()

graingert commented 1 month ago

a nice simple test that shows the problem here:

async def test_ki_protection_check_does_not_freeze_locals() -> None:
    class A:
        pass

    a = A()
    wr_a = weakref.ref(a)
    assert not _core.currently_ki_protected()
    del a
    assert wr_a() is None