Describe the bug
At a moderate to high qubit size (in this case 20 qubits) the Pennylane mixed state simulator allocates too much memory.
To Reproduce
Steps to reproduce the behavior:
Execute one of the pipelines with 20 Qubits.
Expected behavior
N/A
Screenshots
Stacktrace:
Traceback (most recent call last):
File "<path>/disentangling-entanglement-in-qml/.venv/bin/kedro", line 8, in <module>
sys.exit(main())
^^^^^^
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/kedro/framework/cli/cli.py", line 233, in main
cli_collection()
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/kedro/framework/cli/cli.py", line 130, in main
super().main(
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/kedro/framework/cli/project.py", line 225, in run
session.run(
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/kedro/framework/session/session.py", line 395, in run
run_result = runner.run(
^^^^^^^^^^^
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/kedro/runner/runner.py", line 117, in run
self._run(pipeline, catalog, hook_or_null_manager, session_id) # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/kedro/runner/sequential_runner.py", line 75, in _run
run_node(node, catalog, hook_manager, self._is_async, session_id)
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/kedro/runner/runner.py", line 413, in run_node
node = _run_node_sequential(node, catalog, hook_manager, session_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/kedro/runner/runner.py", line 506, in _run_node_sequential
outputs = _call_node_run(
^^^^^^^^^^^^^^^
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/kedro/runner/runner.py", line 472, in _call_node_run
raise exc
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/kedro/runner/runner.py", line 462, in _call_node_run
outputs = node.run(inputs)
^^^^^^^^^^^^^^^^
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/kedro/pipeline/node.py", line 392, in run
raise exc
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/kedro/pipeline/node.py", line 380, in run
outputs = self._run_with_dict(inputs, self._inputs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/kedro/pipeline/node.py", line 437, in _run_with_dict
return self._func(**kwargs)
^^^^^^^^^^^^^^^^^^^^
File "<path>/disentangling-entanglement-in-qml/src/disentangling_entanglement/pipelines/data_generation/nodes.py", line 21, in create_model
return Model(
^^^^^^
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/qml_essentials/model.py", line 149, in __init__
qml.device("default.mixed", shots=self.shots, wires=self.n_qubits),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/pennylane/__init__.py", line 393, in device
dev = plugin_device_class(*args, **options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/pennylane/devices/default_mixed.py", line 220, in __init__
self._state = self._create_basis_state(0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/pennylane/devices/default_mixed.py", line 236, in _create_basis_state
rho = qnp.zeros((2**self.num_wires, 2**self.num_wires), dtype=self.C_DTYPE)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<path>/disentangling-entanglement-in-qml/.venv/lib/python3.11/site-packages/autoray/autoray.py", line 81, in do
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 16.0 TiB for an array with shape (1048576, 1048576) and data type complex128
Additional context
One potential fix could be to use approximations for the mixed state using Pennylane's default mixed state with shots after a certain threshold of qubits. However, we will probably run into a similar problem for the "default_qubit" simulator for a certain amount of qubits as well.
Describe the bug At a moderate to high qubit size (in this case 20 qubits) the Pennylane mixed state simulator allocates too much memory.
To Reproduce Steps to reproduce the behavior: Execute one of the pipelines with 20 Qubits.
Expected behavior N/A
Screenshots Stacktrace:
Additional context One potential fix could be to use approximations for the mixed state using Pennylane's default mixed state with shots after a certain threshold of qubits. However, we will probably run into a similar problem for the "default_qubit" simulator for a certain amount of qubits as well.