Since Qiskit Terra 0.20.0 dense layout and stochastic swap are now multithreaded (via new rust extensions). When running a red queen benchmarks though this causes some issues as red queen will run workers for all the available CPUs on a system and qiskit will spawn a thread pool with the same number of threads. This causes things to bog down as we're running n processes each with n threads so we end up trying to run n^2 CPU bound things in parallel. While this can be dealt with manually either by exporting RAYON_NUM_THREADS=1 or leveraging pytest options to limit concurrency we should think about the interface we want for this and we definitely should provide some guidance on this in the README because it can significantly affect the recorded runtime performance.
Since Qiskit Terra 0.20.0 dense layout and stochastic swap are now multithreaded (via new rust extensions). When running a red queen benchmarks though this causes some issues as red queen will run workers for all the available CPUs on a system and qiskit will spawn a thread pool with the same number of threads. This causes things to bog down as we're running n processes each with n threads so we end up trying to run n^2 CPU bound things in parallel. While this can be dealt with manually either by exporting
RAYON_NUM_THREADS=1or leveraging pytest options to limit concurrency we should think about the interface we want for this and we definitely should provide some guidance on this in the README because it can significantly affect the recorded runtime performance.