generator throw delivered to incorrect generator under trace

[graingert]

Bug report

With the following demo code, a subgenerator yield from from a parent generator main(), that catches exceptions and returns "good":

class Inner:
    def send(self, v):
        return None

    def __iter__(self):
        return self

    def __next__(self):
        return self.send(None)

    def throw(self, *exc_info):
        raise StopIteration("good")

def main():
    return (yield from Inner())

def run(coro):
    coro.send(None)
    try:
        coro.throw(Exception)
    except StopIteration as e:
        print(e.value)

run(main())

when run with python -m trace, the exception is delivered to the outer generator main() instead of being suppressed

graingert@conscientious  ~/projects/cpython   main  ./python -m trace --count -C . demo.py           
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/graingert/projects/cpython/Lib/trace.py", line 741, in <module>
    main()
  File "/home/graingert/projects/cpython/Lib/trace.py", line 729, in main
    t.runctx(code, globs, globs)
  File "/home/graingert/projects/cpython/Lib/trace.py", line 451, in runctx
    exec(cmd, globals, locals)
  File "demo.py", line 26, in <module>
    run(main())
  File "demo.py", line 21, in run
    coro.throw(Exception)
  File "demo.py", line 16, in main
    return (yield from Inner())
            ^^^^^^^^^^^^^^^^^^
Exception
 ✘  graingert@conscientious  ~/projects/cpython   main ± ./python demo.py                      
good

the problem also occurs with the equivalent generator syntax, eg:

def Inner():
    try:
        while True:
            yield None
    except:
        return "good"

Your environment

• CPython versions tested on: c05c31db8c9dfd708b9857bb57f8e5f3ce40266d 3.12b1
• Operating system and architecture: Ubuntu 22.04 x86_64

see also

https://github.com/urllib3/urllib3/issues/3049#issuecomment-1570452951 https://github.com/nedbat/coveragepy/issues/1635

Linked PRs

• gh-105187
• gh-105378

[graingert]

@markshannon bisected to 411b1692811b2ecac59cb0df0f920861c7cf179a

[brandtbucher]

It might be related to the fact that YIELD_VALUE cannot raise...

https://github.com/python/cpython/blob/a99b9d911e0f8cb11b3436bdd8eb649b15d01a50/Python/bytecodes.c#L929-L932

...but INSTRUMENTED_YIELD_VALUE can:

https://github.com/python/cpython/blob/a99b9d911e0f8cb11b3436bdd8eb649b15d01a50/Python/bytecodes.c#L910-L918

That seems incorrect.

[brandtbucher]

This is a little easier for me to follow:

def inner():
    try:
        yield
    except Exception:
        print("caught by inner")
        yield

def outer():
    try:
        yield from inner()
    except Exception:
        print("caught by outer")
        yield

gen = outer()
gen.send(None)
gen.throw(Exception)

brandtbucher@faster-cpython:~/cpython$ ./python bug.py
caught by inner
brandtbucher@faster-cpython:~/cpython$ ./python -m trace -c bug.py
caught by outer

[gaogaotiantian]

Will the actual exception in the trace function be thrown away if we simply remove the error check in instrumented version?

[brandtbucher]

Will the actual exception in the trace function be thrown away if we simply remove the error check in instrumented version?

No, we would need to explicitly clear it.

[Yhg1s]

Is there anything left to do for this issue?

[brandtbucher]

Nope!