Unexpeted reset of the random number generator in QuantumInstance

[David-Kreplin]

Informations

• Qiskit version: 0.37.1
• Python version: 3.10.2 64-bit
• Operating system: Windows 10

The issue is independent of the Qiskit or Python version.

What is the current behavior?

I'm using the QuantumInstance and Qiskit's opflow mechanic for evaluating the expectation value of a parameterized circuit with the QASM simulator. As in the example below, the circuit depends on a single varying parameter and the expectation value is calculated. To my surprise the expectation value show less noise, if the circuit is evaluated separately for each data point instead of all together.

After looking into the QuantumInstance source code, I found that the random number generator is reset to the given seed in each evaluation call. The problem is located in QuantumInstance.assemble() (see https://qiskit.org/documentation/stable/0.37/_modules/qiskit/utils/quantum_instance.html)

def assemble(self, circuits) -> Qobj:
        """assemble circuits"""
        # pylint: disable=cyclic-import
        from qiskit import compiler

        return compiler.assemble(circuits, **self._run_config.to_dict())

Here _run_config resets the simulation seed which is done in each opflow evaluation.

The lower noise level results from the fact that the same random numbers are used in the evaluations for each data point. Since the circuits for closely related data points are also similar, the QASM results are not strongly changing, since the same random numbers are used in the simulation. However, if real random numbers are used (as in the all together evaluation), the finite sampling noise is considerably larger!

Steps to reproduce the problem

Here is a code example for reproducing the issue:

from qiskit import Aer
from qiskit import QuantumCircuit
from qiskit.utils import QuantumInstance
from qiskit.opflow import Z, I
from qiskit.opflow import StateFn,CircuitSampler
from qiskit.opflow import ListOp
import numpy as np
import matplotlib.pyplot as plt

def gen_circuit(x):
    QC = QuantumCircuit(2)
    QC.ry(x,0)
    QC.cnot(0,1)
    return QC

measure_op = Z^I

circuits = ListOp([StateFn(measure_op, is_measurement=True) @ StateFn(gen_circuit(x_)) for x_ in [0.5,0.51,0.52]])

seed = 1234
qinstance = QuantumInstance(Aer.get_backend('qasm_simulator'),
                                 shots=4000,
                                 seed_simulator=seed,
                                 seed_transpiler=seed)
sampler = CircuitSampler(qinstance)

print("QASM in separate calculations")
print(np.real(sampler.convert(circuits[0]).eval()))
print(np.real(sampler.convert(circuits[1]).eval()))
print(np.real(sampler.convert(circuits[2]).eval()))

print("QASM in a single calculation")
print(np.real(sampler.convert(circuits).eval()))

Why this is causing problems can be best seen in a plot, for which the result of the separated evaluation of the circuits yields considerably less noise compared to the single evaluation call. However, the result with more noise is correct, since here, the random number generator is not reset in each evaluation:

x = np.arange(0.5,0.6,0.001)

circ_list = ListOp([StateFn(measure_op, is_measurement=True) @ StateFn(gen_circuit(x_)) for x_ in x])

# All evaluations in a single call
y = np.real(sampler.convert(circ_list).eval())

# All evaluations in separated calls
yy = []
for circ in circ_list:
    yy.append(np.real(sampler.convert(circ).eval()))

plt.figure()
plt.plot(x,yy,label='Eval in separate calls')
plt.plot(x,y,label='Eval in a single call')
plt.legend()

What is the expected behavior?

Both evaluations should yield the same result or at least the same noise level.

Suggested solutions

The solution is unfortunately not so trivial. Probably, the best is to fix the random number generator once in the QuantumInstance initialization.

[woodsp-ibm]

The way seed_simulator works depends on the different simulators. For a list of circuits Aer uses the same random generator seeded once, BasicAer seeds the generator for each circuit in the list. The idea of the seeds was more for reproducibility for testing - yes they behave differently across the simulators when given a list of circuits but either way the outcome is reproducible. Issues have been raised to make the BasicAer behave more like Aer since if you have a list of the same circuit it would be preferred to have different outcomes https://github.com/Qiskit/qiskit-terra/issues/6501

Bottom line this is not something that QuantumInstance can "fix" since it really about the underlying execution behavior of the simulators.

[David-Kreplin]

Thanks for the quick response! I understand that this is not an issue of QuantumInstance, but it sounds for me as a general worrying behavior. For example, if one uses an optimizer based on a finite differences gradient and sets a seed in the beginning, the noise level of the gradient is considerably underestimated, since the random number generator is reinitialized for each finite differences calculation. This yields a bias in the gradient!

Is there any other possibility to initialize the random number generator only in the beginning of my program to achieve reproducible unbiased results?

[woodsp-ibm]

At present the seed gets passed through on each execute, as you have seen. You would like the entire set of executes done using a single seed set up front. I am not aware of a way - the way that backend is setup/configured is normally through backend options, but I have not seen an option to set a seed there. That would be the level it could be set, i.e. set the seed for the simulator backend which is used from there on out. You could check with Aer simulator but a quick look I did not see anything. I really do not know how the random generator is managed there as to whether setting a seed on a first execute and not later ones would reuse the same generator or create a new one with a random seed. @chriseclectic Any thought/ comment.

[jakelishman]

Closing now as this is not in the domain of the metapackage. If issues persist and there's still more to talk about, it seems like the best place would be to open an issue/discussion on Aer.