Greatly simplify variational algorithm workflows

[nonhermitian]

What is the expected enhancement?

The workflow for variational algorithms is very monolithic and not very flexible. Instead it should be possible for users to create variational workflows that are flexible, explicit in what they do, and require minimal knowledge of custom classes. For example, the following works:

# Problem
H2_op = (0.39793742484318045 * I ^ Z) + \
        (-0.39793742484318045 * Z ^ I) + \
        (-0.01128010425623538 * Z ^ Z) + \
        (0.18093119978423156 * X ^ X)

# Ansatz 
ansatz = TwoLocal(H2_op.num_qubits, rotation_blocks='ry',
                  entanglement_blocks='cx')  

# Conversions that I have to do to get useful stuff and should be easy to use functions
op_strings, coeffs = operator_str_and_coeff(H2_op)
meas_circs, masks = opstr_to_meas(op_strings, H2_op.num_qubits)
full_circs = [ansatz.compose(meas_circs[kk]).measure_all(inplace=False) \
              for kk in range(len(meas_circs))]

num_params = ansatz.num_parameters
params = 2*np.pi*np.random.random(opt_params.shape[0])
backend = Aer.get_backend('qasm_simulator')

def vqe_func(params, *args):
    bound_circs = [circ.bind_parameters(params) for circ in full_circs]
    # Vary your shots here if you want.
    job = backend.run(bound_circs, shots=8192)
    counts = job.result().get_counts()
    # Do whatever error mitigation you want here.
    energy = np.sum(coeffs*np.array([expval(counts[kk], masks[kk]) for kk in range(len(masks))]))
    return energy

import scipy.optimize as opt
res = opt.minimize(vqe_func, params, method = 'SLSQP')

Note that I have no need for a quantum_instance, no need to know what mitigation object might be passed, no need for wrapped optimizers, etc. This makes things much much easier to understand, reuse, and extend upon.

[Cryoris]

For completeness, the VQE was actually refactored to simplify this a lot. But I'm not sure if you want to avoid using an algorithm class. So the snippet from above would look like

import numpy as np

from qiskit import Aer
from qiskit.algorithms import VQE
from qiskit.circuit.library import TwoLocal
from qiskit.opflow import I, Z, X
from qiskit.utils import QuantumInstance

# Problem
H2_op = (0.39793742484318045 * I ^ Z) + \
        (-0.39793742484318045 * Z ^ I) + \
        (-0.01128010425623538 * Z ^ Z) + \
        (0.18093119978423156 * X ^ X)

# Ansatz
ansatz = TwoLocal(H2_op.num_qubits, rotation_blocks='ry',
                  entanglement_blocks='cx')

# Initial point and backend
num_params = ansatz.num_parameters
params = 2*np.pi*np.random.random(num_params)
backend = Aer.get_backend('qasm_simulator')
backend.shots = 8291

# Get function to evaluate energy
vqe = VQE(ansatz, quantum_instance=backend)
vqe_func = vqe.get_energy_evaluation(H2_op)

import scipy.optimize as opt
res = opt.minimize(vqe_func, params, method = 'SLSQP')

[nonhermitian]

So that is getting there. Namely the need for custom wrapped optimizers is no longer there. That is nice, as the wrapping never made any sense. However the flexibility is still not there. For example, feedback from people say they want things like the ability to change ansatz mid-execution, or varying the number of shots per function eval. Both of those are easy to accomplish if you allow VQE to be like any other optimization routine and program like I have done above. Then the goal of Qiskit is to provide the building blocks that the user can use inside a function to achieve whatever flavor of vqe (or the like) they want to cook up. I do not think either of these is possible using your modified code above because the VQE class and the quantum_instance do not allow for it. You could add it of course, but then the routines enlarge, and you would just be in the same position the next time a feature request was added.

Supporting things like error mitigation are still problematic. Namely this functionality is hardcoded into the quantum_instance and prohibits people from using whatever they want. In the above the user has access to the counts, and can do whatever they want with them. Similar holds true if someone wanted to do gate based mitigation.

The VQE class itself can still be used if desired, but opening up the core routines so they can be consumed in a more piecemeal fashion would be greatly appreciated by end users. This is going to be especially true in the Qiskit runtime where people want to create a wide variety of variations on the above pattern.

[Cryoris]

Yes agreed. We're working on a design for expectation value class that I think would resolve all above issues though. Some of the main designs we want to include are error mitigation, circuit caching + evaluation for different circuit parameters (so it's suitable for fast evaluation in context of variational algos) and an independence of the quantum instance.

Changing the ansatz mid-circuit is doable with the above, you'll just have to call get_energy_evaluation a second time and use the parameters from the first round of optimization as new initial point. It's for special use-cases like this that we enabled easy access to the loss function.

Also, there's an upcoming refactor to allow arbitrary callables as inputs to VQE (#6381, #6383) which should make it a lot more flexible. 🙂

[nonhermitian]

I think the difference still is that you are hiding functionality in the VQE class. I guess the ask is that the end user be able to construct an entire variational problem using Qiskit building blocks without having to go through the class interface. This enables ultimate flexibility and should not be too much work given that the functionality obviously exists internally, but is tied up in class interfaces.

[Cryoris]

Yes, with the expectation value class we're currently working on that should be an easy feat. I'll make sure we keep this use case in mind 👍🏻

[Cryoris]

Related to #6864.

[nonhermitian]

We have done this so closing