Classical post-processing prior to computing expval

[kharazity]

Feature details

For good reason, the autograd features of pennylane don't work with circuits that conclude with qml.sample(). However, there are many circumstances wherein one would want to do some form classical post-processing such as denoising via a neural network or symmetry post-selection prior to computing expectation values.

Presently, there seems to be no way to perform any classical post-processing on measurement samples that allows one to use the built-in optimization libraries in pennylane. Ideally, there would be an abstraction that would allow one to evaluate a custom loss function (i.e. post-processed expval) and use the result of that loss function to optimize the underlying parameterized quantum circuit from which the samples originated.

In particular, I am interested in the case where a neural network is trained on the sample data and is used to compute the expectation value. This seems like an important abstraction for near term devices where noise/error mitigation schemes can significantly improve algorithm performance.

Implementation

It seems like the implementation for expval is treated entirely by the Device object. As a first thought, we could write a custom_loss method in Device that takes in a lambda function corresponding to the custom loss function specified by the user and a set of arguments corresponding to arguments of the lambda. But I am not sure if this would work.

I'd be interested in learning more about the framework and implementing this as a feature if this is a feasible thing to do.

How important would you say this feature is?

3: Very important! Blocking work.

Additional information

No response

[antalszava]

Hi @kharazity, thank you for posting this issue! :slightly_smiling_face: Indeed, differentiating a QNode that contains qml.sample doesn't work because we don't provide the gradient of such QNodes.

I would imagine custom_loss to work if it takes two functions:

1. The function that computes the loss using the samples as input
2. The gradient of this function

If PennyLane gets both, we can then register function 2. as the gradient of function 1. for each of the interfaces (you've mentioned autograd, though we'll likely want to extend it to the remaining frameworks too: TensorFlow, PyTorch and JAX).

I'd be interested in learning more about the framework and implementing this as a feature if this is a feasible thing to do.

This addition seems to be doable to me, but likely will involve design decisions along the way. I would be curious to get @josh146's thoughts on this issue too.

Overall, it's great to see that there would be a need for such functionality. :slightly_smiling_face:

[trbromley]

Hi @kharazity, this answer from our forum may be relevant to check out here. Could you also consider returning qml.probs() in your QNode, perform the postprocessing, and then average over the probability distribution?

[kharazity]

Does qml.probs() build probability distributions with shots number of samples? Would I need to multiply the distribution by the shot size to get the measurement histogram?

Also, how does expval work when working with results from a "real" device? Ultimately you'd have to estimate expval from the finite samples you get from the device, right? Can we extend this to take any arbitrary (non-negative, real-valued) function of the sample data?

The way my current post-processing method works is to take the observed eigenstates from sampling and then perform a unitary transformation on those statevectors for a set of mutually commuting subgroups of my hamiltonian. I then use the results from each group of measurements to train a neural network to find a representative statevector that can then be used to compute the expectation value of the entire Hamiltonian. Therefore, I need to combine the results from several different experiments into one neural network to compute one expectation value.

From the perspective of autograd, the ML model should just be a blackbox function that takes in samples and spits out a scalar, and wouldn't need to be included in the autodifferentiation step. If g(args) -> [samples], and f([[samples]]) -> scalar, it seems like I should be able to autodiff f(g([args])) without too much difficulty, right?

[josh146]

Does qml.probs() build probability distributions with shots number of samples?

It does, so it will be an estimate of the probability distribution 🙂

So for example, you could do this:

num_shots = 10
dev = qml.device("default.qubit", wires=3, shots=num_shots)

@qml.qnode(dev)
def circuit(weights):
    qml.templates.StronglyEntanglingLayers(weights, wires=[0, 1, 2])
    return qml.probs(wires=[0, 2])

def histogram(weights):
    return circuit(weights) * num_shots

weights = np.random.random([1, 3, 3], requires_grad=True)

Here, histogram returns a vector with the number of times each computational basis appears in the sampled data.

>>> histogram(weights)
[5. 4. 0. 1.]
>>> qml.jacobian(histogram)(weights)
[[[[ 0.5 -3.  -0.5]
   [ 1.  -1.   1. ]
   [ 1.  -2.   1. ]]]
 [[[-0.5  4.   0. ]
   [-1.5 -0.5 -1. ]
   [-1.   0.  -0.5]]]
 [[[ 0.   0.   0. ]
   [ 0.   0.5  0. ]
   [ 0.   0.  -0.5]]]
 [[[ 0.  -1.   0.5]
   [ 0.5  1.   0. ]
   [ 0.   2.   0. ]]]]

More long term, I like an idea that @albi3ro recently proposed. That is:

• Move the statistics post-processing logic out of the Device classes, and into the MeasurementProcess object (a measurement process is created by qml.expval(), qml.var())

• MeasurementProcess objects then provide logic for:

  • Pre-processing of observables
  • Post-processing of state vectors
  • Post-processing of probability distributions/samples

(currently, qml.expval(obs), qml.var(obs), qml.probs() etc. all create a MeasurementProcess object 'under-the-hood').

That way, we could allow PennyLane users to define their own MeasurementProcess objects, that would work on both sampling and statevector devices 🙂 E.g., you could define a custom population_var(obs) function that runs a bespoke measurement post-processing option on the device.

[kharazity]

Even using qml.probs I would still need to train an ML before the expectation value is computed. So is the only problem with my approach that I'm using qml.sample? How does the circuit optimization work when the loss isn't a differentiable function? If the cost function is computed after the qml.probs how does the optimizer know what "direction" to step in?

I think it would be cool if an ML post-processing model could be included in the autodiff stack. It seems like @albi3ro 's idea would lend itself better to this sort of use case. How difficult would it be to include this?

I would be interested in helping to develop a post-processing stack. I have been very impressed with the Pennylane library and I think this use case would be an extremely useful abstraction for near-term algorithm development. I'm not sure anyone else has any classical ML integration in a post-processing layer.

[Jaybsoni]

Even using qml.probs I would still need to train an ML before the expectation value is computed. So is the only problem with my approach that I'm using qml.sample? How does the circuit optimization work when the loss isn't a differentiable function? If the cost function is computed after the qml.probs how does the optimizer know what "direction" to step in?

One idea that you could use to temporary get around this issue would be to include the QNode execution inside your loss function, then you could (manually or otherwise) compute the gradient of the loss function with respect to the weights for your ML application. This could look something like this:

import pennylane as qml
import scipy.optimize as opt
import numpy as np

num_shots = 5
dev = qml.device("default.qubit", wires=2, shots=num_shots)

@qml.qnode(dev)
def circuit(weights):
    qml.RX(weights[0], wires=0)
    qml.RX(weights[0], wires=1)
    qml.RY(weights[1], wires=0)
    qml.RY(weights[1], wires=1)
    return qml.probs(wires=[0, 1])

def histogram(weights):
    return circuit(weights) * num_shots

def loss_function(weights):
    result_hist = np.array(histogram(weights))
    opt_hist = np.array([0., 0., 0., num_shots])

    diff_squares = np.sum((result_hist - opt_hist)**2)
    return diff_squares

def main():
    init_weights = np.random.uniform(0, 2*np.pi, 2)
    print("Init Params: {}".format(init_weights))
    print("Init loss value: {}".format(loss_function(init_weights)))

    res = opt.minimize(loss_function, init_weights)
    print("Optim Params: {}".format(res.x))
    print("Optim Loss value: {}".format(loss_function(res.x)))

    circuit(res.x)
    print(circuit.draw())
    return

if __name__ == "__main__":
    main()

I think it would be cool if an ML post-processing model could be included in the autodiff stack. It seems like @albi3ro 's idea would lend itself better to this sort of use case. How difficult would it be to include this?

I would be interested in helping to develop a post-processing stack. I have been very impressed with the Pennylane library and I think this use case would be an extremely useful abstraction for near-term algorithm development. I'm not sure anyone else has any classical ML integration in a post-processing layer.

It's hard to estimate how difficult such an implementation would be because the scope is still unclear. For example, the change suggested by @albi3ro would require a refactor of the MeasurementProcess class which in itself would be multiple PRs. There are certain internal design decisions which need to be made to facilitate this functionality in a 'clean' manner into PennyLane.

That being said we really appreciate the enthusiasm to contribute! This is a great feature request but this will likely take some time to address. In the meantime you could create a community demo which presents the use case. Once we decide how we want to implement it, we will definitely reach out to you to help contribute!

[kharazity]

Thanks for getting back, I will give this implementation a shot.

I would love to help contribute when you all have made your internal design decisions; I'm just trying to earn my Hacktoberfest shirt!

[CatalinaAlbornoz]

Hi @kharazity, it's great that you're participating in Hacktoberfest! You can find other issues here and remember that you can contribute to other participating projects too.

Thank you for contributing to PennyLane!