9a4854ba-a0a7-4291-945b-b0004bf07198
commented
5 years ago There seems to be a race condition in importlib._bootstrap._ModuleLock that can cause a deadlock. The sequence of operations is as follows:
- Thread 1 calls lock.acquire()
- Thread 1 sets itself as lock.owner and begins importing the module
- Thread 2 calls lock.acquire() and waits for lock.lock
- Thread 2 gets lock.lock
- Thread 1 calls lock.acquire() again, due to a nested import
- Thread 1 sets itself as blocking on lock: _blocking_on[tid1] = lock
- Thread 2 enters lock.has_deadlock()
- Thread 2 busy-waits forever in has_deadlock() because lock.owner == tid1 and _blocking_on[tid1] == lock
- Thread 1 waits forever for lock.lock since thread 2 owns it
The issue was found in pypy3 but it also affects all the recent CPython versions I tried. I can reliably reproduce the issue by adding an artificial delay to _ModuleLock.has_deadlock(), e.g. with this patch:
diff --git a/Lib/test/test_import/__init__.py b/Lib/test/test_import/__init__.py
index f167c84..7f7188e 100644
--- a/Lib/test/test_import/__init__.py
+++ b/Lib/test/test_import/__init__.py
@@ -435,10 +435,15 @@ class ImportTests(unittest.TestCase):
os.does_not_exist
def test_concurrency(self):
+ def delay_has_deadlock(frame, event, arg):
+ if event == 'call' and frame.f_code.co_name == 'has_deadlock':
+ time.sleep(0.2)
+
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'data'))
try:
exc = None
def run():
+ sys.settrace(delay_has_deadlock)
event.wait()
try:
import package
miss-islington
90c69ba2-e16c-4e00-b29e-36bf0e14e605
encukou
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields: ```python assignee = None closed_at = None created_at =
labels = ['3.7', '3.8', 'type-bug', 'library', '3.9']
title = 'Import deadlock detection causes deadlock'
updated_at =
user = 'https://github.com/rlamy'
```
bugs.python.org fields:
```python
activity =
actor = 'vzhestkov'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation =
creator = 'Ronan.Lamy'
dependencies = []
files = []
hgrepos = []
issue_num = 38091
keywords = ['patch']
message_count = 3.0
messages = ['351647', '363232', '397884']
nosy_count = 7.0
nosy_names = ['brett.cannon', 'pitrou', 'eric.snow', 'Ronan.Lamy', 'pconnell', 'miss-islington', 'vzhestkov']
pr_nums = ['17518']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue38091'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8', 'Python 3.9']
```