[ NEW PAGE ] Benchmarking performance

[javabster]

With performance becoming a big feature of qiskit for the rest of the year we should include a page somewhere in docs (not sure where yet) that explains the following:

• what do we mean by "performance" (e.g. speed of transpilation, memory usage, number of cnot gates). Discuss challenges of quantum performance metrics and why they change often (Explain Speed, Quality and Completion)
• why are these metrics important? why should I, a quantum developer, care about these? what value does it add to my quantum program?
• where can I find benchmarking information, or how could I run the benchmarks myself?
• lists of papers relevant to each metric
• small section in latest update page about latest performance improvements?

~Options for where this content could live:~ ~- the "additional resources" tab~ ~- the transpilation specific information could live in the transpile section, not sure about the other information~ ~- if there are multiple pages that could be created maybe it's worth creating a "performance" or "benchmarking" folder in the guides section? Might be confusing though if we also want to put performance discussion interweaved with the other sections on error mitigation or transpilation thought~

update: put "performance" folder in guides section, with at least 2 pages (1x SDK performance, 1x hardware/other performance)

questions to discuss: ~- do we just want to talk about the software performance (e.g. transpile time) or also hardware (quantum volume, CLOPS etc.)~

• how much is SDK specific vs runtime vs transpiler service

[sbrandhsn]

I like this! :-) We should probably also argue why we think the particular benchmark selection is reasonable. I think we should also have a discussion outlining a rust vs python discussion and the associated advantages/drawbacks.