SIGSEGV with generators in free-threaded build

[rostan-t]

Crash report

What happened?

Trying to iterate on a generator from multiple threads under the free-threaded build results in a segmentation fault.

Here is a minimal repro:

import concurrent.futures

def gen():
    while True:
        yield

it = gen()
with concurrent.futures.ThreadPoolExecutor() as executor:
    while True:
        _ = executor.submit(lambda: next(it))

The issue seems to be specific to generators as other kinds of iterators work well in parallel.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.14.0a0 experimental free-threading build (heads/main:c3b6dbff2c, Jun 10 2024, 16:54:16) [GCC 11.4.0]

Linked PRs

• gh-120327

[Fidget-Spinner]

Thanks for the repro. Looking into this.

[Fidget-Spinner]

Confirmed, with pydebug and tsan.

[Zheaoli]

May I take a chance to dive into this issue?

[Fidget-Spinner]

@Zheaoli sorry I assigned myself on this one I have a fix already and will put it up shortly. Do you have any specific issues in mind you would like to work on? I could help mentor/guide you on stuff you're interested in if I can help!

[Zheaoli]

sorry I assigned myself on this one I have a fix already and will put it up shortly

No problem!

Do you have any specific issues in mind you would like to work on? I could help mentor/guide you on stuff you're interested in if I can help!

For now, I'm a fresh man try to be more familiar with the CPython codebase.

I'm now working on #104280 to make dtrace more functional to help debug the process. I'm also interested in JIT and Tier2(especially to enhance the observerability for the Python process). But I'm still trying to find the issue I can fix and the thing I can do(lol

Thanks for knowing that you will be willing to mentor/guide me lol!

[Fidget-Spinner]

@Zheaoli please send me an email I'll help give some pointers privately