SegFault on macOS when accessing globals from multiple threads

[nickovs]

Crash report

Multiple threads attempting to write to a single global variable without a lock can lead to a segmentation fault on macOS.

The script below reproduces the error:

#!/usr/bin/env python

import sys
import time
import threading

print('nogil', hasattr(sys.flags, 'nogil') and sys.flags.nogil)

THREAD_COUNT = 8
BY_HOW_MUCH = 1_000_000

foo = 0
foo_lock = threading.Lock()

def incr(by_how_much):
    global foo
    for i in range(by_how_much):
        with foo_lock:
            foo += 1

def incr2(by_how_much):
    global foo
    for i in range(by_how_much):
        foo += 1

def time_tests(threads):
    start_time = time.time()

    for t in threads:
        t.start()

    for t in threads:
        t.join()

    end_time = time.time()

    print(end_time - start_time)

print("Testing with lock")
threads = [
    threading.Thread(target=incr, args=(BY_HOW_MUCH,))
    for i in range(THREAD_COUNT)
]

foo=0
time_tests(threads)
print(foo)

print("Testing without lock")
threads = [
    threading.Thread(target=incr2, args=(BY_HOW_MUCH,))
    for i in range(THREAD_COUNT)
]

foo=0
time_tests(threads)
print(foo)

The test using a Lock works fine. The test without a lock reliably causes a segmentation fault.

Error messages

Running the above script with a nogil build produces the following output:

nogil True
Testing with lock
18.856343030929565
8000000
Testing without lock
Segmentation fault: 11

Your environment

• CPython versions tested on: Tested with the current nogil-3.12-bugfix branch at ef5bac94da. The same issue occurs with or without configuring --enable-optimizations. The issue does not occur with the 3.12.0b4 beta release.

• Operating system and architecture: macOS 13.4.1, MacBook Pro with quad-core Intel i7.

Notes

The crash can occur with as few as 2 threads, but it shows up more quickly when more threads are used.

While this sort of multiple access is likely to lead to incorrect results, it shouldn't crash the interpreter!