Noisy Qiskit executor with sampling is slow (with change of basis)

[Aaron-Robertson]

Pre-Report Checklist

• [x] I am running the latest version of mitiq
• [x] I checked to make sure that this bug has not already been reported

Issue Description

Resolve todos in test_zne_mitiq_qiskit.py, with focus on executor efficiency of the qiskit_executor() below.

Uncomment this executor in test_zne_mitiq_qiskit.py and run make test-qiskit.

def qiskit_executor(qp: QPROGRAM, shots: int = 500) -> float:
    # initialize a qiskit noise model
    noise_model = NoiseModel()

    # we assume a depolarizing error for each gate of the standard IBM basis
    # set (u1, u2, u3)
    noise_model.add_all_qubit_quantum_error(
        depolarizing_error(BASE_NOISE, 1), ["u1", "u2", "u3"]
    )
    expectation = execute_with_shots_and_noise(
        qp,
        shots=shots,
        obs=ONE_QUBIT_GS_PROJECTOR,
        noise_model=noise_model,
        seed=1,
    )
    return expectation

Test run takes >40 minutes (on test_run_factory_with_number_of_shots and test_mitigate_executor_with_shot_list primarily). As of 03/11, time increase occurs in execution after change of basis with many shots.

Shots	%timeit eigvals, U	%timeit circ.unitary	%timeit circ.measure_all()	%timeit qiskit.execute	%timeit qiskit.execute (no change of basis)
10 ** 4	23.2 µs ± 6.91 µs	286 µs ± 91.4 µs	5.76 ms ± 3.84 ms	1min 3s ± 8.62 s	47.1 ms ± 15 ms
10 ** 5	13.8 µs ± 1.66 µs	179 µs ± 64.8 µs	7.45 ms ± 3.67 ms	12min 55s ± 1min	176 ms ± 5.95 ms
10 ** 6	11.7 µs ± 292 ns	118 µs ± 4.86 µs	3.92 ms ± 2.21 ms	[?]	1.84 s ± 398 ms

Table Note 1: These numbers don't really pass muster because the total test run time is ~40 minutes for a 10 iteration run. These are the timeit run times per loop without our test iterations (so for the >minute default: 1 loop per run, 7 runs). I might argue there's a poor timeit interaction somewhere. Reproduced with similar results on 03/14.

Table Note 2: %timeit 10**6 qiskit.execute failed to complete locally in ~12 hours.

For the %timeit circ.measure_all():

slowest run took 9.24 times longer than the fastest

Test notebook available upon request.

Environment Context

about():

Mitiq: A Python toolkit for implementing error mitigation on quantum computers
==============================================================================
Authored by: Mitiq team, 2020 & later (https://github.com/unitaryfund/mitiq)

Mitiq Version:  0.7.0dev

Core Dependencies
-----------------
Cirq Version:   0.10.0.dev
NumPy Version:  1.20.1
SciPy Version:  1.6.1

Optional Dependencies
---------------------
PyQuil Version: 2.18.0
Qiskit Version: 0.16.4

Python Version: 3.8.5
Platform Info:  Darwin (x86_64)

[Aaron-Robertson]

Note: see #595 for Qiskit version mismatch (it is actually 'qiskit': '0.23.6')

[andreamari]

Thanks @Aaron-Robertson! This is an important issue but I just removed the label bug, since I think this is more a matter of efficiency.

[Aaron-Robertson]

Good point @andreamari, thanks! I clarified the issue accordingly.

Side note: I actually can't edit Mitiq labels—apparently GH only allows label edits with write access. They are attached to issue templates though. In this case I suppose I could have used the feature request template to mark it an enhancement, but that one didn't look quite right based on the text.

[andreamari]

@Aaron-Robertson, there is also a (less visible) option of a blank template. But what you have chosen is totally fine for this issue.

[nathanshammah]

It seems to work fine locally for me actually. I am testing the branch of #584. I uncommented the function qiskit_executor and run the tests, and they work fast for me. A suggestion for #584 would have been to switch from a commented TODO to a pytest.skipdecoration.

---------- coverage: platform darwin, python 3.8.0-final-0 -----------
Name                                          Stmts   Miss  Cover
-----------------------------------------------------------------
mitiq/__init__.py                                 5      0   100%
mitiq/_about.py                                  24     10    58%
mitiq/_typing.py                                  9      0   100%
mitiq/_version.py                                 5      0   100%
mitiq/benchmarks/__init__.py                      1      0   100%
mitiq/benchmarks/maxcut.py                       44     44     0%
mitiq/benchmarks/random_circuits.py              45     45     0%
mitiq/benchmarks/randomized_benchmarking.py      15      3    80%
mitiq/benchmarks/utils.py                        11     11     0%
mitiq/collector.py                               61     41    33%
mitiq/conversions.py                             66     23    65%
mitiq/mitiq_pyquil/__init__.py                    1      1     0%
mitiq/mitiq_pyquil/compiler.py                  149    149     0%
mitiq/mitiq_pyquil/conversions.py                17     17     0%
mitiq/mitiq_pyquil/pyquil_utils.py               29     29     0%
mitiq/mitiq_qiskit/__init__.py                    2      0   100%
mitiq/mitiq_qiskit/conversions.py                94      3    97%
mitiq/mitiq_qiskit/qiskit_utils.py               49      0   100%
mitiq/pec/__init__.py                             4      4     0%
mitiq/pec/pec.py                                 34     34     0%
mitiq/pec/representations/__init__.py             1      1     0%
mitiq/pec/representations/depolarizing.py        73     73     0%
mitiq/pec/sampling.py                            36     36     0%
mitiq/pec/types/__init__.py                       1      1     0%
mitiq/pec/types/types.py                        188    188     0%
mitiq/pec/utils.py                               20     20     0%
mitiq/utils.py                                   71     30    58%
mitiq/zne/__init__.py                             3      0   100%
mitiq/zne/inference.py                          407    205    50%
mitiq/zne/scaling/__init__.py                     3      0   100%
mitiq/zne/scaling/folding.py                    213    119    44%
mitiq/zne/scaling/parameter.py                   48     33    31%
mitiq/zne/zne.py                                 29      5    83%
-----------------------------------------------------------------
TOTAL                                          1758   1125    36%
Coverage XML written to file coverage.xml

======================================= 77 passed in 28.70s ========================================
  ~/gi/mitiq   Aaron-Robertson-qiskit-utils *5 !2 ❯ python                      30s  mitiqdev
Python 3.8.0 (default, Nov  6 2019, 15:49:01)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import mitiq
mitiq.about()>>> mitiq.about()

Mitiq: A Python toolkit for implementing error mitigation on quantum computers
==============================================================================
Authored by: Mitiq team, 2020 & later (https://github.com/unitaryfund/mitiq)

Mitiq Version:  0.7.0dev

Core Dependencies
-----------------
Cirq Version:   0.10.0
NumPy Version:  1.20.1
SciPy Version:  1.4.1

Optional Dependencies
---------------------
PyQuil Version: 2.28.0
Qiskit Version: 0.24.0

Python Version: 3.8.0
Platform Info:  Darwin (x86_64)

[Aaron-Robertson]

@nathanshammah @rmlarose Aha! Nathan is correct above and on the community call this morning. Submitted a PR, not sure if it can be absorbed into milestone 0.7.0, but it's a minor change.

Testing again with a test notebook, I see the slowness is actually present in the old higher shot counts so that could be worth more investigation. PR should probably be unlinked.

[rmlarose]

So, this is fixed by having qiskit_version>=0.24? Or does anything else need to be done?

[Aaron-Robertson]

That wasn't it actually. "Fix" was the test changes with fewer shots. This issue of a slower execute with a change of basis, vs without, is still present and visible for high shot counts. It doesn't impact our tests anymore though, so could close the issue if that's the priority.

[grmlarose]

This will be closed by #623 with new observables.

[github-actions[bot]]

This issue had no activity for 2 months, and will be closed in one week unless there is new activity. Cheers!