Quantum Software Benchmarks

DISCLAIMER: the benchmark results are only update to date till 2020, a rewrite of the benchmark scripts are still in progress

We benchmark several popular quantum computation softwares/frameworks/simulators to test their performance in practical daily research.

This benchmark is written with the following principles:

• try to use the user interface (UI) instead of high performance hack from internals.
• focus on laptop runnable scale (4-25 qubits) instead of High Performance Cluster scale

Results

The complete results are included in RESULTS.md

Installation

You should be able to run this benchmark with the bin/benchmark script.

    Quantum Circuit Simulation Benchmark

install                 install dependencies
run [package]           run benchmark of [package] or run all benchmarks by default
parallel [package]      spawn [package] benchmark in a process or run all benchmark in parallel by default
help                    print this message

Requirements

• Conda
• Python 3
• Julia 1.0+
• CUDA Toolkit (optional) for benchmarking CUDA backend

Make sure you have the above required dependency installed, then run the following script, which will install all the dependencies for you in a new environment.

bin/benchmark install

Run Benchmark

bin/benchmark benchmark

This will usually take a few hours to finish, thus if you want to run it in the backend, you can use parallel instead of benchmark. Afterwards, you can generate the plot by running:

bin/plot

Contributing

Please feel free to update and add new benchmarks. Check CONTRIBUTING.md for more details.

Acknowledge

We thank the following people for reviewing our benchmarks:

• Juan Gomez, Christopher J. Wood for reviewing qiskit
• Craig Gidney for reviewing Cirq
• corryvrequan for reviewing qulacs
• Damian Steiger for reviewing ProjectQ
• johannesjmeyer for reviewing PennyLane

License

MIT license