unitaryfund / mitiq

Mitiq is an open source toolkit for implementing error mitigation techniques on most current intermediate-scale quantum computers.
https://mitiq.readthedocs.io
GNU General Public License v3.0
363 stars 161 forks source link

Speed up and/or remove redundant unit tests #1427

Closed Aaron-Robertson closed 1 year ago

Aaron-Robertson commented 2 years ago

Pre-Request Checklist

Issue Description

Local test times are onerous for Mitiq contributors and for the repository actions. Contribution would be easier if tests run faster.

Based on my own local testing, the 15 slowest tests are Time Test
99.69s mitiq/pec/tests/test_pec.py::test_pec_decorator_qiskit
82.92s mitiq/zne/tests/test_zne.py::test_with_observable_two_qubits[sample_bitstrings]
49.32s mitiq/pec/tests/test_pec.py::test_qiskit_noiseless_decomposition_multiqubit[1]
45.16s mitiq/pec/tests/test_pec.py::test_qiskit_noiseless_decomposition_multiqubit[2]
30.31s mitiq/zne/tests/test_zne.py::test_with_observable_batched_factory[sample_bitstrings]
28.99s mitiq/zne/tests/test_zne.py::test_with_observable_adaptive_factory[sample_bitstrings]
26.77s mitiq/cdr/tests/test_cdr.py::test_mitigated_execute_with_cdr[5-kwargs1-linear_fit_function-pennylane]
26.55s mitiq/cdr/tests/test_cdr.py::test_mitigated_execute_with_cdr[5-kwargs1-linear_fit_function_no_intercept-pennylane]
26.50s mitiq/cdr/tests/test_cdr.py::test_mitigated_execute_with_cdr[5-kwargs0-linear_fit_function_no_intercept-pennylane]
25.77s mitiq/cdr/tests/test_cdr.py::test_mitigated_execute_with_cdr[5-kwargs0-linear_fit_function_no_intercept-qiskit]
25.31s mitiq/cdr/tests/test_cdr.py::test_mitigated_execute_with_cdr[3-kwargs1-linear_fit_function-pennylane]
24.93s mitiq/cdr/tests/test_cdr.py::test_mitigated_execute_with_cdr[3-kwargs0-linear_fit_function_no_intercept-pennylane]
24.74s mitiq/cdr/tests/test_cdr.py::test_mitigated_execute_with_cdr[5-kwargs0-linear_fit_function-qiskit]
24.38s mitiq/cdr/tests/test_cdr.py::test_mitigated_execute_with_cdr[4-kwargs0-linear_fit_function-pennylane]
24.03s mitiq/cdr/tests/test_cdr.py::test_mitigated_execute_with_cdr[5-kwargs1-linear_fit_function_no_intercept-qiskit]

Proposed Solution

Look for redundancy in the above tests and remove/reduce wherever possible. Potentially split this issue into two to three sub issues, with the first focused on CDR and the final focused on PyQuil.

Additional References

Add the --durations=N flag to a test run to check local runtimes, where N is the number of ranked test length to track (and 0 returns all test times ranked). I.e. to obtain the above data, the flag is --durations=15 and the full command is pytest -n auto -v --cov=mitiq --cov-report=term --cov-report=xml --ignore=mitiq/interface/mitiq_pyquil --durations=15.

natestemen commented 1 year ago

Some previous work to speed up tests: https://github.com/unitaryfund/mitiq/pull/583

I am working on this, just wanted to add that since it's related.

Misty-W commented 1 year ago

thanks for working on this @natestemen! For purposes of the release I'll close this issue and create a new one for incremental improvements in a future milestone.