Closed mtreinish closed 2 years ago
mtreinish
commented
2 years ago I'm running this locally now (it's quite slow compared to qiskit and tweedledum) the only thing that I still need to verify works is gate counting at the end and that we're returning a valid swap cx count
mtreinish
commented
2 years ago It looks like it's working correctly (except for the hard coded layout method string). The results locally are:
Benchmark: 54QBT_900CYC_QSE_5.qasm
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Name ┃ Min ┃ Max ┃ Mean ┃ cx ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ tket │ 187 (1.0) │ 190.3 (1.0) │ 188.4 (1.0) │ 1.207e+04 │
│ (GraphPlaceme… │ │ │ │ (1.0) │
│ + Routing) │ │ │ │ │
│ tket │ 231.3 (1.24) │ 235.4 (1.24) │ 233.3 (1.24) │ 1.7e+04 (1.41) │
│ (GraphPlaceme… │ │ │ │ │
│ + Routing) │ │ │ │ │
└────────────────┴──────────────┴──────────────┴──────────────┴────────────────┘
Benchmark: 54QBT_900CYC_QSE_6.qasm
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Name ┃ Min ┃ Max ┃ Mean ┃ cx ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ tket │ 46.79 (1.0) │ 47.54 (1.0) │ 47.02 (1.0) │ 3126 (1.0) │
│ (GraphPlaceme… │ │ │ │ │
│ + Routing) │ │ │ │ │
│ tket │ 94.46 (2.02) │ 111.7 (2.35) │ 99.63 (2.12) │ 1.09e+04 │
│ (GraphPlaceme… │ │ │ │ (3.49) │
│ + Routing) │ │ │ │ │
└────────────────┴──────────────┴──────────────┴──────────────┴────────────────┘
Benchmark: 54QBT_900CYC_QSE_7.qasm
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Name ┃ Min ┃ Max ┃ Mean ┃ cx ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ tket │ 22.92 (1.0) │ 23.73 (1.0) │ 23.35 (1.0) │ 672 (1.0) │
│ (GraphPlacem… │ │ │ │ │
│ + Routing) │ │ │ │ │
│ tket │ 177.2 (7.73) │ 279.6 (11.78) │ 227.7 (9.75) │ 2.029e+04 │
│ (GraphPlacem… │ │ │ │ (30.20) │
│ + Routing) │ │ │ │ │
└───────────────┴──────────────┴───────────────┴──────────────┴────────────────┘
Benchmark: 54QBT_900CYC_QSE_8.qasm
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Name ┃ Min ┃ Max ┃ Mean ┃ cx ┃
┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ tket │ 9.994 (1.0) │ 10.14 (1.0) │ 10.09 (1.0) │ 279 (1.0) │
│ (GraphPlacem… │ │ │ │ │
│ + Routing) │ │ │ │ │
│ tket │ 198.1 (19.82) │ 205.5 (20.27) │ 203.3 (20.16) │ 1.459e+04 │
│ (GraphPlacem… │ │ │ │ (52.30) │
│ + Routing) │ │ │ │ │
└───────────────┴───────────────┴───────────────┴───────────────┴──────────────┘
Benchmark: 54QBT_900CYC_QSE_9.qasm
┏━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Name ┃ Min ┃ Max ┃ Mean ┃ cx ┃
┡━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ tket │ 22.62 (1.0) │ 22.79 (1.0) │ 22.7 (1.0) │ 909 (1.0) │
│ (GraphPlacement │ │ │ │ │
│ + Routing) │ │ │ │ │
│ tket │ 192 (8.49) │ 197.8 (8.68) │ 195.3 (8.60) │ 1.518e+04 │
│ (GraphPlacement │ │ │ │ (16.70) │
│ + Routing) │ │ │ │ │
└─────────────────┴─────────────┴──────────────┴──────────────┴────────────────┘
boschmitt
commented
2 years ago LGTM.
This commit adds a new benchmark for testing tket's performance on the queko benchmark suite. It runs queko with two different layout (placement in the tket terminology) algorithms and the default routing/mapping pass. The only place where these benchmarks are less than ideal is that to get things to work correctly I needed to copy each circuit in the timed function. Without this the inplace mutation and shared references was causing the benchmarks to fail. Hopefully a better long term solution can be found eventually.