Closed garrison closed 6 months ago
I keep on adding to the list above, but so far this only seems to happen on Python 3.8 on Ubuntu.
I noticed a warning when running the test suite under Python 3.12 that may be a clue.
test/cutting/test_cutting_workflows.py: 48 warnings
test/cutting/test_cutting_roundtrip.py: 420 warnings
/usr/local/lib/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning: This process (pid=17907) is multi-threaded, use of fork() may lead to deadlocks in the child.
self.pid = os.fork()
I was initially confused because we are not using multiprocessing explicitly, but it turns out that Qiskit's transpiler actually uses multiprocessing when a pass manager is asked to transpile multiple circuits in the same call. Indeed, the traceback of the warning is
circuit_knitting/cutting/cutting_experiments.py:188: in generate_cutting_experiments
def test_generate_cutting_experiments(self):
with self.subTest("simple circuit and observable"):
qc = QuantumCircuit(2)
qc.append(
TwoQubitQPDGate(QPDBasis.from_instruction(CXGate()), label="cut_cx"),
qargs=[0, 1],
)
comp_coeffs = [
(0.5, WeightType.EXACT),
(0.5, WeightType.EXACT),
(0.5, WeightType.EXACT),
(-0.5, WeightType.EXACT),
(0.5, WeightType.EXACT),
(-0.5, WeightType.EXACT),
]
> subexperiments, coeffs = generate_cutting_experiments(
qc, PauliList(["ZZ"]), np.inf
)
test/cutting/test_cutting_experiments.py:52:
subexperiments_dict[label] = pass_manager.run(subexperiments)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.tox/py312/lib/python3.12/site-packages/qiskit/transpiler/passmanager.py:420: in wrapper
circuit_knitting/cutting/cutting_experiments.py:188: in generate_cutting_experiments
return meth(*meth_args, **meth_kwargs)
.tox/py312/lib/python3.12/site-packages/qiskit/transpiler/passmanager.py:182: in run
subexperiments_dict[label] = pass_manager.run(subexperiments)
.tox/py312/lib/python3.12/site-packages/qiskit/transpiler/passmanager.py:420: in wrapper
return super().run(
.tox/py312/lib/python3.12/site-packages/qiskit/passmanager/passmanager.py:246: in run
return meth(*meth_args, **meth_kwargs)
.tox/py312/lib/python3.12/site-packages/qiskit/transpiler/passmanager.py:182: in run
return parallel_map(
.tox/py312/lib/python3.12/site-packages/qiskit/utils/parallel.py:171: in parallel_map
return super().run(
.tox/py312/lib/python3.12/site-packages/qiskit/passmanager/passmanager.py:246: in run
raise error
.tox/py312/lib/python3.12/site-packages/qiskit/utils/parallel.py:161: in parallel_map
return parallel_map(
.tox/py312/lib/python3.12/site-packages/qiskit/utils/parallel.py:171: in parallel_map
future = executor.map(_task_wrapper, param)
/usr/local/lib/python3.12/concurrent/futures/process.py:822: in map
raise error
.tox/py312/lib/python3.12/site-packages/qiskit/utils/parallel.py:161: in parallel_map
results = super().map(partial(_process_chunk, fn),
/usr/local/lib/python3.12/concurrent/futures/_base.py:608: in map
future = executor.map(_task_wrapper, param)
/usr/local/lib/python3.12/concurrent/futures/process.py:822: in map
fs = [self.submit(fn, *args) for args in zip(*iterables)]
/usr/local/lib/python3.12/concurrent/futures/process.py:794: in submit
results = super().map(partial(_process_chunk, fn),
/usr/local/lib/python3.12/concurrent/futures/_base.py:608: in map
self._start_executor_manager_thread()
/usr/local/lib/python3.12/concurrent/futures/process.py:733: in _start_executor_manager_thread
fs = [self.submit(fn, *args) for args in zip(*iterables)]
/usr/local/lib/python3.12/concurrent/futures/process.py:794: in submit
self._launch_processes()
/usr/local/lib/python3.12/concurrent/futures/process.py:760: in _launch_processes
self._start_executor_manager_thread()
/usr/local/lib/python3.12/concurrent/futures/process.py:733: in _start_executor_manager_thread
self._spawn_process()
/usr/local/lib/python3.12/concurrent/futures/process.py:770: in _spawn_process
self._launch_processes()
/usr/local/lib/python3.12/concurrent/futures/process.py:760: in _launch_processes
p.start()
/usr/local/lib/python3.12/multiprocessing/process.py:121: in start
self._spawn_process()
/usr/local/lib/python3.12/concurrent/futures/process.py:770: in _spawn_process
self._popen = self._Popen(self)
/usr/local/lib/python3.12/multiprocessing/context.py:282: in _Popen
p.start()
/usr/local/lib/python3.12/multiprocessing/process.py:121: in start
return Popen(process_obj)
/usr/local/lib/python3.12/multiprocessing/popen_fork.py:19: in __init__
self._popen = self._Popen(self)
/usr/local/lib/python3.12/multiprocessing/context.py:282: in _Popen
self._launch(process_obj)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <multiprocessing.popen_fork.Popen object at 0x7fafdf377710>
process_obj = <ForkProcess name='ForkProcess-1' parent=6471 unknown>
return Popen(process_obj)
/usr/local/lib/python3.12/multiprocessing/popen_fork.py:19: in __init__
def _launch(self, process_obj):
code = 1
parent_r, child_w = os.pipe()
child_r, parent_w = os.pipe()
> self.pid = os.fork()
which points to exactly the line that is hanging under nbmake
in the tests under Python 3.8.
(I wonder if the issue here might be in part due to interaction with pytest and/or nbmake, but it's doesn't appear like nbmake uses any multiprocessing directly.)
So perhaps this is again an instance of https://pythonspeed.com/articles/python-multiprocessing/, in which it could perhaps be fixed by using mp.get_context("spawn")
in Qiskit.
Possibly relevant issues
Another way we could potentially work around this is by setting QISKIT_NUM_PROCS=1
.
The symptom seems to have been fixed by performing the ubuntu tests on Python 3.9 instead of 3.8. With #556, the generate_cutting_experiments
function will no longer use the pass manager (and therefore not need multiprocessing), so we should no longer see this even on Python 3.8 once that PR is merged.
Now that #514 is merged, we can actually see which notebook hangs intermittently on CI.
Full log at: