"negative refcount on merged object" (or silent) segfault

[ppolewicz]

incrementor.py

import sys
print('nogil', hasattr(sys.flags, 'nogil') and sys.flags.nogil)

import threading

THREAD_COUNT = 3
BY_HOW_MUCH = 1_000_000

class Incrementor:
    def __init__(self):
        self.c = 0

def incr(incrementor, by_how_much):
    for i in range(by_how_much):
        incrementor.c += 1

incrementor = Incrementor()

threads = [
    threading.Thread(target=incr, args=(incrementor, BY_HOW_MUCH))
    for i in range(THREAD_COUNT)
]

for t in threads:
    t.start()

for t in threads:
    t.join()

print(incrementor.c)

Dockerfile

FROM colesbury/python-nogil
#FROM python:3.9-slim

WORKDIR /usr/src/app
COPY incrementor.py .
CMD ["python3", "incrementor.py"]

it runs fine on 3.9, but crashes on 3.9-nogil:

$ docker build -t incrementor .
Sending build context to Docker daemon  4.608kB
Step 1/4 : FROM colesbury/python-nogil
 ---> d3ae3b21184c
Step 2/4 : WORKDIR /usr/src/app
 ---> Using cache
 ---> 64cbd60184dd
Step 3/4 : COPY incrementor.py .
 ---> 3415c53a4f12
Step 4/4 : CMD ["python3", "incrementor.py"]
 ---> Running in 924cb429e162
Removing intermediate container 924cb429e162
 ---> 77c6053ae938
Successfully built 77c6053ae938
Successfully tagged incrementor:latest
$ docker run -it --rm --name incrementor-running incrementor
nogil True
Fatal Python error: negative refcount on merged object
Python runtime state: initialized

Thread 0x00007f81977fe700 (most recent call first):
  File "/usr/local/lib/python3.9/threading.py", line 929 in _bootstrap_inner
  File "/usr/local/lib/python3.9/threading.py", line 881 in run
  File "/usr/local/lib/python3.9/threading.py", line 937 in _bootstrap_inner
  File "/usr/local/lib/python3.9/threading.py", line 907 in _bootstrap

Thread 0x00007f8197fff700 (most recent call first):
  File "/usr/src/app/incrementor.py", line 17 in incr
  File "/usr/local/lib/python3.9/threading.py", line 881 in run
  File "/usr/local/lib/python3.9/threading.py", line 937 in _bootstrap_inner
  File "/usr/local/lib/python3.9/threading.py", line 907 in _bootstrap

Current thread 0x00007f819cc37700 (most recent call first):
  File "/usr/src/app/incrementor.py", line 17 in incr
  File "/usr/local/lib/python3.9/threading.py", line 881 in run
  File "/usr/local/lib/python3.9/threading.py", line 937 in _bootstrap_inner
  File "/usr/local/lib/python3.9/threading.py", line 907 in _bootstrap

Thread 0x00007f819cc8d740 (most recent call first):
  File "/usr/local/lib/python3.9/threading.py", line 991 in join
  File "/usr/src/app/incrementor.py", line 30 in <module>
Press any key to continue...

but sometimes it just prints

nogil True

and exits with code 139 (Segmentation Fault)

[colesbury]

Please use the base image nogil/python. The colesbury/python-nogil image is out of date. I thought I had removed all references to that from the documentation and readme.

Let me know if you still see the crash when building from nogil/python.

[ppolewicz]

I confirm that it no longer crashes when using nogil/python