[BUG] Returning non-commuting observables and probabilities from a QNode raises an error

[eddddddy]

Expected behavior

The below code should print out the quantities:

def debug():
    dev = qml.device("default.qubit", wires=2)

    @qml.qnode(dev)
    def circuit(x):
        qml.RY(x, wires=0)
        qml.CNOT(wires=[0, 1])
        return qml.expval(qml.PauliX(0) @ qml.PauliZ(1)), qml.expval(qml.PauliZ(0) @ qml.PauliY(1)), qml.probs(wires=[0, 1])

    x = np.array(0.5)
    print(circuit(x))

Actual behavior

An error is raised instead:

Traceback (most recent call last):
  File "C:\Users\Edward\Documents\pennylane\pennylane\tape\tape.py", line 157, in expand_tape
    rotations, diag_obs = qml.grouping.diagonalize_qwc_pauli_words(
  File "C:\Users\Edward\Documents\pennylane\pennylane\grouping\transformations.py", line 140, in diagonalize_qwc_pauli_words
    raise ValueError("The list of Pauli words are not qubit-wise commuting.")
ValueError: The list of Pauli words are not qubit-wise commuting.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "../scripts/adjoint_diff.py", line 54, in <module>
    main()
  File "../scripts/adjoint_diff.py", line 50, in main
    debug()
  File "../scripts/adjoint_diff.py", line 18, in debug
    print(circuit(x))
  File "C:\Users\Edward\Documents\pennylane\pennylane\qnode.py", line 661, in __call__
    res = qml.execute(
  File "C:\Users\Edward\Documents\pennylane\pennylane\interfaces\execution.py", line 370, in execute
    qml.interfaces.cache_execute(
  File "C:\Users\Edward\Documents\pennylane\pennylane\interfaces\execution.py", line 197, in wrapper
    res = fn(execution_tapes.values(), **kwargs)
  File "C:\Users\Edward\Documents\pennylane\pennylane\interfaces\execution.py", line 121, in fn
    tapes = [expand_fn(tape) for tape in tapes]
  File "C:\Users\Edward\Documents\pennylane\pennylane\interfaces\execution.py", line 121, in <listcomp>
    tapes = [expand_fn(tape) for tape in tapes]
  File "C:\Users\Edward\Documents\pennylane\pennylane\interfaces\execution.py", line 351, in <lambda>
    expand_fn = lambda tape: device.expand_fn(tape, max_expansion=max_expansion)
  File "C:\Users\Edward\Documents\pennylane\pennylane\_device.py", line 688, in expand_fn
    return self.default_expand_fn(circuit, max_expansion=max_expansion)
  File "C:\Users\Edward\Documents\pennylane\pennylane\_device.py", line 663, in default_expand_fn
    circuit = circuit.expand(depth=max_expansion, stop_at=self.stopping_condition)
  File "C:\Users\Edward\Documents\pennylane\pennylane\tape\tape.py", line 674, in expand
    new_tape = expand_tape(
  File "C:\Users\Edward\Documents\pennylane\pennylane\tape\tape.py", line 161, in expand_tape
    raise qml.QuantumFunctionError(
pennylane.QuantumFunctionError: Only observables that are qubit-wise commuting Pauli words can be returned on the same wire

Additional information

The above error only occurs when two non-commuting observables are returned in conjunction with probabilities.

If we instead return non-commuting observables, e.g. return qml.expval(qml.PauliX(0) @ qml.PauliZ(1)), qml.expval(qml.PauliZ(0) @ qml.PauliY(1)), the error disappears.

If we instead return a single observable and probabilities, e.g. return qml.expval(qml.PauliX(0) @ qml.PauliZ(1)), qml.probs(wires=[0, 1]), the error also disappears.

This occurs in the old return type system.

Source code

def debug():
    dev = qml.device("default.qubit", wires=2)

    @qml.qnode(dev)
    def circuit(x):
        qml.RY(x, wires=0)
        qml.CNOT(wires=[0, 1])

        # this works
        # return qml.expval(qml.PauliX(0) @ qml.PauliZ(1)), qml.expval(qml.PauliZ(0) @ qml.PauliY(1))

        # this works too
        # return qml.expval(qml.PauliX(0) @ qml.PauliZ(1)), qml.probs(wires=[0, 1])

        # this doesn't
        # return qml.expval(qml.PauliX(0) @ qml.PauliZ(1)), qml.expval(qml.PauliZ(0) @ qml.PauliY(1)), qml.probs(wires=[0, 1])

    x = np.array(0.5)
    print(circuit(x))

Tracebacks

Traceback (most recent call last):
  File "C:\Users\Edward\Documents\pennylane\pennylane\tape\tape.py", line 157, in expand_tape
    rotations, diag_obs = qml.grouping.diagonalize_qwc_pauli_words(
  File "C:\Users\Edward\Documents\pennylane\pennylane\grouping\transformations.py", line 140, in diagonalize_qwc_pauli_words
    raise ValueError("The list of Pauli words are not qubit-wise commuting.")
ValueError: The list of Pauli words are not qubit-wise commuting.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "../scripts/adjoint_diff.py", line 54, in <module>
    main()
  File "../scripts/adjoint_diff.py", line 50, in main
    debug()
  File "../scripts/adjoint_diff.py", line 18, in debug
    print(circuit(x))
  File "C:\Users\Edward\Documents\pennylane\pennylane\qnode.py", line 661, in __call__
    res = qml.execute(
  File "C:\Users\Edward\Documents\pennylane\pennylane\interfaces\execution.py", line 370, in execute
    qml.interfaces.cache_execute(
  File "C:\Users\Edward\Documents\pennylane\pennylane\interfaces\execution.py", line 197, in wrapper
    res = fn(execution_tapes.values(), **kwargs)
  File "C:\Users\Edward\Documents\pennylane\pennylane\interfaces\execution.py", line 121, in fn
    tapes = [expand_fn(tape) for tape in tapes]
  File "C:\Users\Edward\Documents\pennylane\pennylane\interfaces\execution.py", line 121, in <listcomp>
    tapes = [expand_fn(tape) for tape in tapes]
  File "C:\Users\Edward\Documents\pennylane\pennylane\interfaces\execution.py", line 351, in <lambda>
    expand_fn = lambda tape: device.expand_fn(tape, max_expansion=max_expansion)
  File "C:\Users\Edward\Documents\pennylane\pennylane\_device.py", line 688, in expand_fn
    return self.default_expand_fn(circuit, max_expansion=max_expansion)
  File "C:\Users\Edward\Documents\pennylane\pennylane\_device.py", line 663, in default_expand_fn
    circuit = circuit.expand(depth=max_expansion, stop_at=self.stopping_condition)
  File "C:\Users\Edward\Documents\pennylane\pennylane\tape\tape.py", line 674, in expand
    new_tape = expand_tape(
  File "C:\Users\Edward\Documents\pennylane\pennylane\tape\tape.py", line 161, in expand_tape
    raise qml.QuantumFunctionError(
pennylane.QuantumFunctionError: Only observables that are qubit-wise commuting Pauli words can be returned on the same wire

System information

Name: PennyLane
Version: 0.26.0.dev0
Summary: PennyLane is a Python quantum machine learning library by Xanadu Inc.
Home-page: https://github.com/XanaduAI/pennylane
Author:
Author-email:
License: Apache License 2.0
Location: c:\python38\lib\site-packages
Requires: appdirs, autograd, autoray, cachetools, networkx, numpy, pennylane-lightning, retworkx, scipy, semantic-version, toml
Required-by: PennyLane-Lightning

Platform info:           Windows-10-10.0.19041-SP0
Python version:          3.8.8
Numpy version:           1.22.3
Scipy version:           1.7.0
Installed devices:
- default.gaussian (PennyLane-0.26.0.dev0)
- default.mixed (PennyLane-0.26.0.dev0)
- default.qubit (PennyLane-0.26.0.dev0)
- default.qubit.autograd (PennyLane-0.26.0.dev0)
- default.qubit.jax (PennyLane-0.26.0.dev0)
- default.qubit.tf (PennyLane-0.26.0.dev0)
- default.qubit.torch (PennyLane-0.26.0.dev0)
- default.qutrit (PennyLane-0.26.0.dev0)
- lightning.qubit (PennyLane-Lightning-0.25.0)

Existing GitHub issues

• [X] I have searched existing GitHub issues to make sure the issue does not already exist.

[Jaybsoni]

After doing some digging, I have updates regarding the cause of this error and some suggestions for fixes.

The error message is raised in ~/tape/tape.py when we do some validation checks to determine if the tape can be executed correctly:

Looking at the observables one would agree that this error message is correct. Things get a little more complicated when we remove the qml.probs measurement. After removing it, we are somehow able to correctly execute the tape and return a tuple of the results.

This happens because we apply a batch transform to our tapes before we execute them (line 338 in execution.py):

There is logic in the device batch transform which tries to address simultaneous measurements of non-commuting observables by splitting up the tape into multiple tapes with each measurement grouped (line 759 in _device.py):

Im not sure why exactly, but as long as probs, counts, sample, all counts are not measured in the tape, we are allowed to do this splitting (maybe @josh146 has some ideas as to why this is the case). Once the tapes are split, the error message does not get triggered because each tape is executed separately, and so we don't run into the issue of simultaneously measuring non-commuting observables.

It is also important to note, @Qottmann has worked on a tape transform to allow simultaneous measurements of non-commuting observables. This seems like a case where multiple people wanted to include this functionality in, and we don't have a standard way of dealing with it yet.

Now the question is, what makes sense for the correct behaviour here?

In the long term, I would love to see a push to standardize approach to how we validate tapes and handle non-commuting measurements (both on hardware and on simulator).

In the short-term, I think we can provide more detail by listing out the measurements in the error message so users can see what exactly is being measured together would be useful.

Let me know what you think about this. We could also add a check that loops through the tape and determines if we are measuring any one of(probs, counts, sample, all counts) with expectation values, and use that info to trigger a different error message.