`multiprocessing.pool.ThreadPool` crash

[denballakh]

Crash report

This code crashes on my machine:

import multiprocessing.pool
import itertools
objs = [0]
def func():
    objs[0] += 1
for _ in map(print, itertools.count()):  # fancy way to print numbers :)
    with multiprocessing.pool.ThreadPool(2) as pool:
        pool.apply_async(func)
        pool.apply_async(func)

Same idea, but crashes faster:

import multiprocessing.pool
import itertools
objs = [[0] for _ in range(1000)]
def func():
    for p in objs:
        p[0] += 1 # it can be any positive number (so p[0] eventually reaches >256 range, so it is no more interned)
for _ in map(print, itertools.count()):  # fancy way to print numbers :)
    with multiprocessing.pool.ThreadPool(2) as pool:
        pool.apply_async(func)
        pool.apply_async(func)

Error messages

None, I see only numbers, and then process eventually crashes.

Your environment

• CPython versions tested on: Python 3.12.0a4 (heads/nogil-3.12:4526c07cae, Jun 20 2023, 21:00:08) [MSC v.1936 64 bit (AMD64)] on win32, i built it myself an hour ago
• Operating system and architecture: Windows, 64-bit, AMD

  >>> sys.getwindowsversion()
  sys.getwindowsversion(major=10, minor=0, build=19045, platform=2, service_pack='')
  >>> platform.architecture()
  ('64bit', 'WindowsPE')
  >>> platform.machine()
  'AMD64'
  >>> platform.platform()
  'Windows-10-10.0.19045-SP0'
  >>> platform.processor()
  'Intel64 Family 6 Model 142 Stepping 12, GenuineIntel'
  >>> platform.python_branch()
  'heads/nogil-3.12'
  >>> platform.python_build()
  ('heads/nogil-3.12:4526c07cae', 'Jun 20 2023 21:00:08')
  >>> platform.python_compiler()
  'MSC v.1936 64 bit (AMD64)'

[denballakh]

I rebuilt interpreter in debug mode (build.bat -c Debug) and got some results. Now i am running interpreter with flags -X dev -X faulthandler. I am running second version of my code (which contains objs = [[0] for _ in range(1000)]).

In all cases this window appears:

If i click "Retry":

130
Assertion failed: refcount >= 0, file D:\GitHub\cpython\nogil-3.12\Objects\object.c, line 2543
131
...
178
Windows fatal exception: code 0x80000003

Stack (most recent call first):
  File "D:\GitHub\openSR\nogilbug\opensr.py", line 6 in func
  File "D:\GitHub\cpython\nogil-3.12\Lib\multiprocessing\pool.py", line 125 in worker
  File "D:\GitHub\cpython\nogil-3.12\Lib\threading.py", line 965 in run
  File "D:\GitHub\cpython\nogil-3.12\Lib\threading.py", line 1014 in _bootstrap_inner
  File "D:\GitHub\cpython\nogil-3.12\Lib\threading.py", line 985 in _bootstrap

If i click "Abort" or "Continue":

129
Assertion failed: refcount >= 0, file D:\GitHub\cpython\nogil-3.12\Objects\object.c, line 2543
130
...
178
Fatal Python error: Aborted

Stack (most recent call first):
  File "D:\GitHub\openSR\nogilbug\opensr.py", line 6 in func
  File "D:\GitHub\cpython\nogil-3.12\Lib\multiprocessing\pool.py", line 125 in worker
  File "D:\GitHub\cpython\nogil-3.12\Lib\threading.py", line 965 in run
  File "D:\GitHub\cpython\nogil-3.12\Lib\threading.py", line 1014 in _bootstrap_inner
  File "D:\GitHub\cpython\nogil-3.12\Lib\threading.py", line 985 in _bootstrap

Sometimes stack can be printed before "Assertion failed", and i can see something like this:

130
Windows fatal exception: access violation

Stack (most recent call first):
131  File "
D:\GitHub\openSR\nogilbug\opensr.py", line 6 in func
  File "D:\GitHub\cpython\nogil-3.12\Lib\multiprocessing\pool.py", line 125 in worker
  File "D:\GitHub\cpython\nogil-3.12\Lib\threading.py", line 965 in run
  File "D:\GitHub\cpython\nogil-3.12\Lib\threading.py", line 1014 in _bootstrap_inner
  File "D:\GitHub\cpython\nogil-3.12\Lib\threading.py", line 985 in _bootstrap
132
133
134
Assertion failed: refcount >= 0, file D:\GitHub\cpython\nogil-3.12\Objects\object.c, line 2543
135

Sometimes this crazy thing happens:

129
Assertion failed: new_shared >= 0 && "negative refcount", file D:\GitHub\cpython\nogil-3.12\Objects\object.c, line 2601
130
131
132Assertion failed: new_shared >= 0 && "negative refcount", file D:\GitHub\cpython\nogil-3.12\Objects\object.c, line 2601

133
Windows fatal exception: 134access violation

Stack (most recent call first):
  File "D:\GitHub\openSR\nogilbug\opensr.py", line 6 in func
  File "D:\GitHub\cpython\nogil-3.12\Lib\multiprocessing\pool.py", line 125 in worker
  File "D:\GitHub\cpython\nogil-3.12\Lib\threading.py", line 965 in run
  File "D:\GitHub\cpython\nogil-3.12\Lib\threading.py", line 1014 in _bootstrap_inner
  File 135"
Assertion failed: refcount >= 0, file D:\GitHub\cpython\nogil-3.12\Objects\object.c, line 2543
D:\GitHub\cpython\nogil-3.12\Lib\threading.py", line 985 in _bootstrap
136

[colesbury]

Thanks for the bug report @denballakh. I should have a fix for this soon.