issues
search
QuantumSavory
/
QuantumClifford.jl
Clifford circuits, graph states, and other quantum Stabilizer formalism tools.
MIT License
103
stars
42
forks
source link
Misc cleanup and fixes, noticed when preparing the decoder wiki
#262
Closed
Krastanov
closed
2 months ago
github-actions[bot]
commented
2 months ago
Benchmark Result
# Judge result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmarks: - Target: 18 Apr 2024 - 15:07 - Baseline: 18 Apr 2024 - 15:14 * Package commits: - Target: 65fcd1 - Baseline: 64f695 * Julia commits: - Target: 0f7674 - Baseline: 0f7674 * Julia command flags: - Target: None - Baseline: None * Environment variables: - Target: None - Baseline: None ## Results A ratio greater than `1.0` denotes a possible regression (marked with :x:), while a ratio less than `1.0` denotes a possible improvement (marked with :white_check_mark:). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds). | ID | time ratio | memory ratio | |--------------------------------------------------------------------|------------------------------|------------------------------| | `["circuitsim", "mctrajectories_sumtype", "q1001_r1"]` | 1.07 (5%) :x: | 1.00 (1%) | | `["circuitsim", "mctrajectories_union", "q1001_r1"]` | 1.06 (5%) :x: | 1.00 (1%) | | `["circuitsim", "pftrajectories", "q101_r1"]` | 0.95 (5%) :white_check_mark: | 1.00 (1%) | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 0.14 (5%) :white_check_mark: | 0.98 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 0.44 (5%) :white_check_mark: | 0.83 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 0.35 (5%) :white_check_mark: | 0.83 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 0.43 (5%) :white_check_mark: | 0.83 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 0.98 (5%) | 0.97 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 0.99 (5%) | 0.98 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 0.86 (5%) :white_check_mark: | 0.90 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 0.84 (5%) :white_check_mark: | 0.91 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 0.87 (5%) :white_check_mark: | 0.92 (1%) :white_check_mark: | | `["pauli", "mul", "20000000"]` | 1.07 (5%) :x: | 1.00 (1%) | | `["stabilizer", "tensor", "diag_pow5_20"]` | 1.07 (5%) :x: | 1.00 (1%) | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ### Target ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 2467 s 0 s 187 s 6679 s 0 s #2 0 MHz 2552 s 0 s 159 s 6616 s 0 s #3 0 MHz 2576 s 0 s 188 s 6575 s 0 s #4 0 MHz 1738 s 0 s 195 s 7404 s 0 s Memory: 15.606494903564453 GB (13354.48046875 MB free) Uptime: 937.94 sec Load Avg: 1.05 1.06 0.78 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` ### Baseline ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 3109 s 0 s 286 s 10132 s 0 s #2 0 MHz 3373 s 0 s 241 s 9907 s 0 s #3 0 MHz 3648 s 0 s 276 s 9611 s 0 s #4 0 MHz 3504 s 0 s 294 s 9736 s 0 s Memory: 15.606494903564453 GB (13194.15625 MB free) Uptime: 1358.95 sec Load Avg: 1.03 1.09 0.93 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Target result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmark: 18 Apr 2024 - 15:7 * Package commit: 65fcd1 * Julia commit: 0f7674 * Julia command flags: None * Environment variables: None ## Results Below is a table of this job's results, obtained by running the benchmarks. The values listed in the `ID` column have the structure `[parent_group, child_group, ..., key]`, and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero. | ID | time | GC time | memory | allocations | |--------------------------------------------------------------------|----------------:|----------:|----------------:|------------:| | `["circuitsim", "mctrajectories", "q1001_r1"]` | 15.667 ms (5%) | | 500.77 KiB (1%) | 18019 | | `["circuitsim", "mctrajectories", "q101_r1"]` | 170.619 μs (5%) | | 50.53 KiB (1%) | 1818 | | `["circuitsim", "mctrajectories_sumtype", "q1001_r1"]` | 14.108 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_sumtype", "q101_r1"]` | 117.570 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "mctrajectories_union", "q1001_r1"]` | 13.851 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_union", "q101_r1"]` | 116.217 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "pftrajectories", "q1001_r1"]` | 64.560 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r100"]` | 177.432 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r10000"]` | 1.100 ms (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q101_r1"]` | 6.473 μs (5%) | | 9.42 KiB (1%) | 201 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r1"]` | 44.173 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r100"]` | 155.281 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]` | 1.086 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]` | 5.838 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q101_r1"]` | 4.518 μs (5%) | | 9.47 KiB (1%) | 202 | | `["circuitsim", "pftrajectories_union", "q1001_r1"]` | 22.612 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r100"]` | 131.627 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r10000"]` | 1.053 ms (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q101_r1"]` | 2.344 μs (5%) | | 96 bytes (1%) | 2 | | `["clifford", "dense", "cnot250_on_dense500_destab"]` | 11.594 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_dense500_stab"]` | 5.801 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_destab"]` | 1.123 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_stab"]` | 490.437 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot_on_dense500_destab"]` | 50.916 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_dense500_stab"]` | 23.183 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_destab"]` | 26.309 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_stab"]` | 13.365 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "dense500_on_dense500_destab"]` | 11.568 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_dense500_stab"]` | 5.805 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_destab"]` | 980.163 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_stab"]` | 490.277 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "symbolic", "cnot250_on_dense500_destab"]` | 1.440 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_dense500_stab"]` | 688.117 μs (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_destab"]` | 1.160 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_stab"]` | 569.645 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_destab"]` | 4.749 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_stab"]` | 2.304 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_destab"]` | 4.688 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_stab"]` | 2.294 μs (5%) | | | | | `["ecc", "evaluate_decoder", "shor_bp_comm"]` | 1.859 ms (5%) | | 1.77 MiB (1%) | 42581 | | `["ecc", "evaluate_decoder", "shor_bp_naivesyn"]` | 4.165 ms (5%) | | 3.61 MiB (1%) | 81592 | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 4.566 ms (5%) | | 3.71 MiB (1%) | 82902 | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 25.345 ms (5%) | | 3.97 MiB (1%) | 116082 | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 50.255 ms (5%) | | 7.97 MiB (1%) | 227768 | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 51.436 ms (5%) | | 8.06 MiB (1%) | 228854 | | `["ecc", "evaluate_decoder", "shor_table_comm"]` | 259.135 μs (5%) | | 241.42 KiB (1%) | 4593 | | `["ecc", "evaluate_decoder", "shor_table_naivesyn"]` | 868.234 μs (5%) | | 334.20 KiB (1%) | 4790 | | `["ecc", "evaluate_decoder", "shor_table_shorsyn"]` | 1.268 ms (5%) | | 427.73 KiB (1%) | 5876 | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 688.833 ms (5%) | 20.256 ms | 169.85 MiB (1%) | 1018690 | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 1.409 s (5%) | 41.903 ms | 347.87 MiB (1%) | 2107672 | | `["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]` | 1.410 s (5%) | 39.383 ms | 348.54 MiB (1%) | 2129271 | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 401.165 ms (5%) | | 7.31 MiB (1%) | 135571 | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 825.480 ms (5%) | | 16.45 MiB (1%) | 304138 | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 842.126 ms (5%) | | 18.96 MiB (1%) | 336552 | | `["ecc", "evaluate_decoder", "toric8_pymatch_comm"]` | 4.250 ms (5%) | | 1.95 MiB (1%) | 23176 | | `["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]` | 12.670 ms (5%) | | 5.58 MiB (1%) | 79355 | | `["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]` | 21.257 ms (5%) | | 8.09 MiB (1%) | 111769 | | `["ecc", "evaluate_decoder", "toric8_table_comm"]` | 2.651 ms (5%) | | 1.64 MiB (1%) | 23082 | | `["ecc", "evaluate_decoder", "toric8_table_naivesyn"]` | 10.185 ms (5%) | | 4.93 MiB (1%) | 79160 | | `["ecc", "evaluate_decoder", "toric8_table_shorsyn"]` | 18.745 ms (5%) | | 7.44 MiB (1%) | 111574 | | `["pauli", "mul", "100"]` | 14.415 ns (5%) | | | | | `["pauli", "mul", "1000"]` | 18.218 ns (5%) | | | | | `["pauli", "mul", "100000"]` | 722.244 ns (5%) | | | | | `["pauli", "mul", "20000000"]` | 222.988 μs (5%) | | | | | `["stabilizer", "canon", "cano500"]` | 3.081 ms (5%) | | | | | `["stabilizer", "canon", "diag_cano500"]` | 676.466 μs (5%) | | | | | `["stabilizer", "canon", "diag_gott500"]` | 4.147 ms (5%) | | 5.19 MiB (1%) | 34521 | | `["stabilizer", "canon", "diag_rref500"]` | 665.755 μs (5%) | | | | | `["stabilizer", "canon", "gott500"]` | 6.606 ms (5%) | | 5.19 MiB (1%) | 34531 | | `["stabilizer", "canon", "md_cano500"]` | 1.592 ms (5%) | | | | | `["stabilizer", "canon", "md_rref500"]` | 1.632 ms (5%) | | | | | `["stabilizer", "canon", "rref500"]` | 3.144 ms (5%) | | | | | `["stabilizer", "project", "destabilizer"]` | 18.214 μs (5%) | | 288 bytes (1%) | 5 | | `["stabilizer", "project", "stabilizer"]` | 9.067 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "tensor", "diag_pow5_20"]` | 1.527 ms (5%) | | 23.97 MiB (1%) | 34 | | `["stabilizer", "tensor", "pow5_20"]` | 2.836 μs (5%) | | 6.44 KiB (1%) | 31 | | `["stabilizer", "trace", "destabilizer"]` | 34.424 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "trace", "stabilizer"]` | 24.175 μs (5%) | | 112 bytes (1%) | 3 | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 2467 s 0 s 187 s 6679 s 0 s #2 0 MHz 2552 s 0 s 159 s 6616 s 0 s #3 0 MHz 2576 s 0 s 188 s 6575 s 0 s #4 0 MHz 1738 s 0 s 195 s 7404 s 0 s Memory: 15.606494903564453 GB (13354.48046875 MB free) Uptime: 937.94 sec Load Avg: 1.05 1.06 0.78 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Baseline result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmark: 18 Apr 2024 - 15:14 * Package commit: 64f695 * Julia commit: 0f7674 * Julia command flags: None * Environment variables: None ## Results Below is a table of this job's results, obtained by running the benchmarks. The values listed in the `ID` column have the structure `[parent_group, child_group, ..., key]`, and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero. | ID | time | GC time | memory | allocations | |--------------------------------------------------------------------|----------------:|----------:|----------------:|------------:| | `["circuitsim", "mctrajectories", "q1001_r1"]` | 14.983 ms (5%) | | 500.77 KiB (1%) | 18019 | | `["circuitsim", "mctrajectories", "q101_r1"]` | 171.541 μs (5%) | | 50.53 KiB (1%) | 1818 | | `["circuitsim", "mctrajectories_sumtype", "q1001_r1"]` | 13.181 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_sumtype", "q101_r1"]` | 118.942 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "mctrajectories_union", "q1001_r1"]` | 13.107 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_union", "q101_r1"]` | 117.891 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "pftrajectories", "q1001_r1"]` | 67.395 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r100"]` | 175.468 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r10000"]` | 1.145 ms (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q101_r1"]` | 6.833 μs (5%) | | 9.42 KiB (1%) | 201 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r1"]` | 44.704 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r100"]` | 159.598 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]` | 1.135 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]` | 5.820 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q101_r1"]` | 4.488 μs (5%) | | 9.47 KiB (1%) | 202 | | `["circuitsim", "pftrajectories_union", "q1001_r1"]` | 22.602 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r100"]` | 136.385 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r10000"]` | 1.102 ms (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q101_r1"]` | 2.344 μs (5%) | | 96 bytes (1%) | 2 | | `["clifford", "dense", "cnot250_on_dense500_destab"]` | 11.639 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_dense500_stab"]` | 5.793 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_destab"]` | 1.124 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_stab"]` | 490.456 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot_on_dense500_destab"]` | 51.136 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_dense500_stab"]` | 23.404 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_destab"]` | 26.690 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_stab"]` | 13.374 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "dense500_on_dense500_destab"]` | 11.609 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_dense500_stab"]` | 5.766 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_destab"]` | 980.322 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_stab"]` | 490.237 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "symbolic", "cnot250_on_dense500_destab"]` | 1.441 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_dense500_stab"]` | 687.545 μs (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_destab"]` | 1.161 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_stab"]` | 570.315 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_destab"]` | 4.728 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_stab"]` | 2.304 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_destab"]` | 4.659 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_stab"]` | 2.314 μs (5%) | | | | | `["ecc", "evaluate_decoder", "shor_bp_comm"]` | 1.847 ms (5%) | | 1.77 MiB (1%) | 42581 | | `["ecc", "evaluate_decoder", "shor_bp_naivesyn"]` | 4.165 ms (5%) | | 3.61 MiB (1%) | 81648 | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 32.173 ms (5%) | 26.267 ms | 3.79 MiB (1%) | 84708 | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 57.151 ms (5%) | | 4.76 MiB (1%) | 144207 | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 145.566 ms (5%) | | 9.57 MiB (1%) | 284018 | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 118.722 ms (5%) | | 9.66 MiB (1%) | 285104 | | `["ecc", "evaluate_decoder", "shor_table_comm"]` | 254.355 μs (5%) | | 241.42 KiB (1%) | 4593 | | `["ecc", "evaluate_decoder", "shor_table_naivesyn"]` | 877.901 μs (5%) | | 334.20 KiB (1%) | 4790 | | `["ecc", "evaluate_decoder", "shor_table_shorsyn"]` | 1.278 ms (5%) | | 427.73 KiB (1%) | 5876 | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 703.965 ms (5%) | 15.404 ms | 175.59 MiB (1%) | 1052444 | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 1.420 s (5%) | 30.498 ms | 355.25 MiB (1%) | 2151086 | | `["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]` | 1.410 s (5%) | 36.070 ms | 350.22 MiB (1%) | 2139113 | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 464.203 ms (5%) | | 8.11 MiB (1%) | 163696 | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 978.670 ms (5%) | | 18.04 MiB (1%) | 360388 | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 963.926 ms (5%) | | 20.56 MiB (1%) | 392802 | | `["ecc", "evaluate_decoder", "toric8_pymatch_comm"]` | 4.267 ms (5%) | | 1.95 MiB (1%) | 23176 | | `["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]` | 12.732 ms (5%) | | 5.58 MiB (1%) | 79355 | | `["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]` | 21.370 ms (5%) | | 8.09 MiB (1%) | 111769 | | `["ecc", "evaluate_decoder", "toric8_table_comm"]` | 2.639 ms (5%) | | 1.64 MiB (1%) | 23082 | | `["ecc", "evaluate_decoder", "toric8_table_naivesyn"]` | 10.456 ms (5%) | | 4.93 MiB (1%) | 79160 | | `["ecc", "evaluate_decoder", "toric8_table_shorsyn"]` | 19.077 ms (5%) | | 7.44 MiB (1%) | 111574 | | `["pauli", "mul", "100"]` | 14.505 ns (5%) | | | | | `["pauli", "mul", "1000"]` | 18.820 ns (5%) | | | | | `["pauli", "mul", "100000"]` | 732.911 ns (5%) | | | | | `["pauli", "mul", "20000000"]` | 209.321 μs (5%) | | | | | `["stabilizer", "canon", "cano500"]` | 3.081 ms (5%) | | | | | `["stabilizer", "canon", "diag_cano500"]` | 676.744 μs (5%) | | | | | `["stabilizer", "canon", "diag_gott500"]` | 4.131 ms (5%) | | 5.19 MiB (1%) | 34521 | | `["stabilizer", "canon", "diag_rref500"]` | 665.774 μs (5%) | | | | | `["stabilizer", "canon", "gott500"]` | 6.521 ms (5%) | | 5.19 MiB (1%) | 34531 | | `["stabilizer", "canon", "md_cano500"]` | 1.606 ms (5%) | | | | | `["stabilizer", "canon", "md_rref500"]` | 1.649 ms (5%) | | | | | `["stabilizer", "canon", "rref500"]` | 3.147 ms (5%) | | | | | `["stabilizer", "project", "destabilizer"]` | 18.425 μs (5%) | | 288 bytes (1%) | 5 | | `["stabilizer", "project", "stabilizer"]` | 9.217 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "tensor", "diag_pow5_20"]` | 1.422 ms (5%) | | 23.97 MiB (1%) | 34 | | `["stabilizer", "tensor", "pow5_20"]` | 2.719 μs (5%) | | 6.44 KiB (1%) | 31 | | `["stabilizer", "trace", "destabilizer"]` | 34.304 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "trace", "stabilizer"]` | 24.135 μs (5%) | | 112 bytes (1%) | 3 | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 3109 s 0 s 286 s 10132 s 0 s #2 0 MHz 3373 s 0 s 241 s 9907 s 0 s #3 0 MHz 3648 s 0 s 276 s 9611 s 0 s #4 0 MHz 3504 s 0 s 294 s 9736 s 0 s Memory: 15.606494903564453 GB (13194.15625 MB free) Uptime: 1358.95 sec Load Avg: 1.03 1.09 0.93 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Runtime information | Runtime Info | | |:--|:--| | BLAS #threads | 2 | | `BLAS.vendor()` | `lbt` | | `Sys.CPU_THREADS` | 4 | `lscpu` output: Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 48 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Vendor ID: AuthenticAMD Model name: AMD EPYC 7763 64-Core Processor CPU family: 25 Model: 1 Thread(s) per core: 2 Core(s) per socket: 2 Socket(s): 1 Stepping: 1 BogoMIPS: 4890.86 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm Virtualization: AMD-V Hypervisor vendor: Microsoft Virtualization type: full L1d cache: 64 KiB (2 instances) L1i cache: 64 KiB (2 instances) L2 cache: 1 MiB (2 instances) L3 cache: 32 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-3 Vulnerability Gather data sampling: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode Vulnerability Spec store bypass: Vulnerable Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected | Cpu Property | Value | |:------------------ |:---------------------------------------------------------- | | Brand | AMD EPYC 7763 64-Core Processor | | Vendor | :AMD | | Architecture | :Unknown | | Model | Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00 | | Cores | 16 physical cores, 16 logical cores (on executing CPU) | | | No Hyperthreading hardware capability detected | | Clock Frequencies | Not supported by CPU | | Data Cache | Level 1:3 : (32, 512, 32768) kbytes | | | 64 byte cache line size | | Address Size | 48 bits virtual, 48 bits physical | | SIMD | 256 bit = 32 byte max. SIMD vector size | | Time Stamp Counter | TSC is accessible via `rdtsc` | | | TSC runs at constant rate (invariant from clock frequency) | | Perf. Monitoring | Performance Monitoring Counters (PMC) are not supported | | Hypervisor | Yes, Microsoft |
Benchmark Result
# Judge result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmarks: - Target: 18 Apr 2024 - 15:15 - Baseline: 18 Apr 2024 - 15:22 * Package commits: - Target: be8388 - Baseline: 64f695 * Julia commits: - Target: 0f7674 - Baseline: 0f7674 * Julia command flags: - Target: None - Baseline: None * Environment variables: - Target: None - Baseline: None ## Results A ratio greater than `1.0` denotes a possible regression (marked with :x:), while a ratio less than `1.0` denotes a possible improvement (marked with :white_check_mark:). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds). | ID | time ratio | memory ratio | |--------------------------------------------------------------------|------------------------------|------------------------------| | `["circuitsim", "pftrajectories", "q1001_r1"]` | 0.95 (5%) :white_check_mark: | 1.00 (1%) | | `["circuitsim", "pftrajectories", "q101_r1"]` | 0.95 (5%) :white_check_mark: | 1.00 (1%) | | `["circuitsim", "pftrajectories_sumtype", "q1001_r1"]` | 0.73 (5%) :white_check_mark: | 1.00 (1%) | | `["circuitsim", "pftrajectories_sumtype", "q1001_r100"]` | 0.94 (5%) :white_check_mark: | 1.00 (1%) | | `["circuitsim", "pftrajectories_sumtype", "q101_r1"]` | 0.73 (5%) :white_check_mark: | 1.00 (1%) | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 0.92 (5%) :white_check_mark: | 0.98 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 0.38 (5%) :white_check_mark: | 0.83 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 0.33 (5%) :white_check_mark: | 0.83 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 0.42 (5%) :white_check_mark: | 0.83 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 0.97 (5%) | 1.02 (1%) :x: | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 1.04 (5%) | 1.02 (1%) :x: | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 0.80 (5%) :white_check_mark: | 0.90 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 0.81 (5%) :white_check_mark: | 0.91 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 0.85 (5%) :white_check_mark: | 0.92 (1%) :white_check_mark: | | `["stabilizer", "trace", "destabilizer"]` | 1.06 (5%) :x: | 1.00 (1%) | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ### Target ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 1779 s 0 s 162 s 7262 s 0 s #2 0 MHz 2054 s 0 s 172 s 6979 s 0 s #3 0 MHz 2571 s 0 s 188 s 6450 s 0 s #4 0 MHz 2907 s 0 s 186 s 6108 s 0 s Memory: 15.606494903564453 GB (13357.609375 MB free) Uptime: 925.16 sec Load Avg: 1.14 1.07 0.76 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` ### Baseline ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 2528 s 0 s 261 s 10527 s 0 s #2 0 MHz 3405 s 0 s 265 s 9654 s 0 s #3 0 MHz 3625 s 0 s 267 s 9440 s 0 s #4 0 MHz 3969 s 0 s 264 s 9093 s 0 s Memory: 15.606494903564453 GB (13249.9453125 MB free) Uptime: 1338.58 sec Load Avg: 1.15 1.12 0.92 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Target result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmark: 18 Apr 2024 - 15:15 * Package commit: be8388 * Julia commit: 0f7674 * Julia command flags: None * Environment variables: None ## Results Below is a table of this job's results, obtained by running the benchmarks. The values listed in the `ID` column have the structure `[parent_group, child_group, ..., key]`, and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero. | ID | time | GC time | memory | allocations | |--------------------------------------------------------------------|----------------:|----------:|----------------:|------------:| | `["circuitsim", "mctrajectories", "q1001_r1"]` | 15.111 ms (5%) | | 500.77 KiB (1%) | 18019 | | `["circuitsim", "mctrajectories", "q101_r1"]` | 169.545 μs (5%) | | 50.53 KiB (1%) | 1818 | | `["circuitsim", "mctrajectories_sumtype", "q1001_r1"]` | 12.961 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_sumtype", "q101_r1"]` | 118.391 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "mctrajectories_union", "q1001_r1"]` | 13.136 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_union", "q101_r1"]` | 117.288 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "pftrajectories", "q1001_r1"]` | 59.882 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r100"]` | 180.045 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r10000"]` | 1.148 ms (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q101_r1"]` | 6.031 μs (5%) | | 9.42 KiB (1%) | 201 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r1"]` | 44.072 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r100"]` | 158.475 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]` | 1.130 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]` | 5.821 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q101_r1"]` | 4.448 μs (5%) | | 9.47 KiB (1%) | 202 | | `["circuitsim", "pftrajectories_union", "q1001_r1"]` | 22.611 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r100"]` | 136.694 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r10000"]` | 1.099 ms (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q101_r1"]` | 2.344 μs (5%) | | 96 bytes (1%) | 2 | | `["clifford", "dense", "cnot250_on_dense500_destab"]` | 11.496 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_dense500_stab"]` | 5.734 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_destab"]` | 1.121 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_stab"]` | 560.591 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot_on_dense500_destab"]` | 50.153 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_dense500_stab"]` | 24.395 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_destab"]` | 26.219 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_stab"]` | 13.755 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "dense500_on_dense500_destab"]` | 11.509 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_dense500_stab"]` | 5.740 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_destab"]` | 1.123 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_stab"]` | 560.612 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "symbolic", "cnot250_on_dense500_destab"]` | 1.446 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_dense500_stab"]` | 687.718 μs (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_destab"]` | 1.158 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_stab"]` | 569.930 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_destab"]` | 4.739 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_stab"]` | 2.304 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_destab"]` | 4.708 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_stab"]` | 2.304 μs (5%) | | | | | `["ecc", "evaluate_decoder", "shor_bp_comm"]` | 1.812 ms (5%) | | 1.77 MiB (1%) | 42609 | | `["ecc", "evaluate_decoder", "shor_bp_naivesyn"]` | 4.123 ms (5%) | | 3.62 MiB (1%) | 81648 | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 4.539 ms (5%) | | 3.70 MiB (1%) | 82636 | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 24.597 ms (5%) | | 3.97 MiB (1%) | 116082 | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 49.884 ms (5%) | | 7.97 MiB (1%) | 227768 | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 50.023 ms (5%) | | 8.06 MiB (1%) | 228854 | | `["ecc", "evaluate_decoder", "shor_table_comm"]` | 254.523 μs (5%) | | 241.42 KiB (1%) | 4593 | | `["ecc", "evaluate_decoder", "shor_table_naivesyn"]` | 864.997 μs (5%) | | 334.20 KiB (1%) | 4790 | | `["ecc", "evaluate_decoder", "shor_table_shorsyn"]` | 1.265 ms (5%) | | 427.73 KiB (1%) | 5876 | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 675.963 ms (5%) | 17.018 ms | 172.23 MiB (1%) | 1032711 | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 1.423 s (5%) | 32.888 ms | 350.38 MiB (1%) | 2122463 | | `["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]` | 1.443 s (5%) | 35.364 ms | 356.70 MiB (1%) | 2177228 | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 386.739 ms (5%) | | 7.31 MiB (1%) | 135571 | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 805.859 ms (5%) | | 16.45 MiB (1%) | 304138 | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 811.335 ms (5%) | | 18.96 MiB (1%) | 336552 | | `["ecc", "evaluate_decoder", "toric8_pymatch_comm"]` | 4.273 ms (5%) | | 1.95 MiB (1%) | 23176 | | `["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]` | 12.547 ms (5%) | | 5.58 MiB (1%) | 79355 | | `["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]` | 21.021 ms (5%) | | 8.09 MiB (1%) | 111769 | | `["ecc", "evaluate_decoder", "toric8_table_comm"]` | 2.616 ms (5%) | | 1.64 MiB (1%) | 23082 | | `["ecc", "evaluate_decoder", "toric8_table_naivesyn"]` | 10.164 ms (5%) | | 4.93 MiB (1%) | 79160 | | `["ecc", "evaluate_decoder", "toric8_table_shorsyn"]` | 18.616 ms (5%) | | 7.44 MiB (1%) | 111574 | | `["pauli", "mul", "100"]` | 14.395 ns (5%) | | | | | `["pauli", "mul", "1000"]` | 17.918 ns (5%) | | | | | `["pauli", "mul", "100000"]` | 722.756 ns (5%) | | | | | `["pauli", "mul", "20000000"]` | 192.749 μs (5%) | | | | | `["stabilizer", "canon", "cano500"]` | 3.082 ms (5%) | | | | | `["stabilizer", "canon", "diag_cano500"]` | 676.368 μs (5%) | | | | | `["stabilizer", "canon", "diag_gott500"]` | 4.178 ms (5%) | | 5.19 MiB (1%) | 34521 | | `["stabilizer", "canon", "diag_rref500"]` | 665.897 μs (5%) | | | | | `["stabilizer", "canon", "gott500"]` | 6.580 ms (5%) | | 5.19 MiB (1%) | 34531 | | `["stabilizer", "canon", "md_cano500"]` | 1.541 ms (5%) | | | | | `["stabilizer", "canon", "md_rref500"]` | 1.585 ms (5%) | | | | | `["stabilizer", "canon", "rref500"]` | 3.140 ms (5%) | | | | | `["stabilizer", "project", "destabilizer"]` | 18.475 μs (5%) | | 288 bytes (1%) | 5 | | `["stabilizer", "project", "stabilizer"]` | 8.887 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "tensor", "diag_pow5_20"]` | 1.257 ms (5%) | | 23.97 MiB (1%) | 34 | | `["stabilizer", "tensor", "pow5_20"]` | 2.769 μs (5%) | | 6.44 KiB (1%) | 31 | | `["stabilizer", "trace", "destabilizer"]` | 33.192 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "trace", "stabilizer"]` | 24.455 μs (5%) | | 112 bytes (1%) | 3 | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 1779 s 0 s 162 s 7262 s 0 s #2 0 MHz 2054 s 0 s 172 s 6979 s 0 s #3 0 MHz 2571 s 0 s 188 s 6450 s 0 s #4 0 MHz 2907 s 0 s 186 s 6108 s 0 s Memory: 15.606494903564453 GB (13357.609375 MB free) Uptime: 925.16 sec Load Avg: 1.14 1.07 0.76 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Baseline result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmark: 18 Apr 2024 - 15:22 * Package commit: 64f695 * Julia commit: 0f7674 * Julia command flags: None * Environment variables: None ## Results Below is a table of this job's results, obtained by running the benchmarks. The values listed in the `ID` column have the structure `[parent_group, child_group, ..., key]`, and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero. | ID | time | GC time | memory | allocations | |--------------------------------------------------------------------|----------------:|----------:|----------------:|------------:| | `["circuitsim", "mctrajectories", "q1001_r1"]` | 15.121 ms (5%) | | 500.77 KiB (1%) | 18019 | | `["circuitsim", "mctrajectories", "q101_r1"]` | 170.857 μs (5%) | | 50.53 KiB (1%) | 1818 | | `["circuitsim", "mctrajectories_sumtype", "q1001_r1"]` | 13.512 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_sumtype", "q101_r1"]` | 118.721 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "mctrajectories_union", "q1001_r1"]` | 13.205 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_union", "q101_r1"]` | 117.318 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "pftrajectories", "q1001_r1"]` | 63.258 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r100"]` | 173.763 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r10000"]` | 1.103 ms (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q101_r1"]` | 6.381 μs (5%) | | 9.42 KiB (1%) | 201 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r1"]` | 60.432 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r100"]` | 169.104 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]` | 1.104 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]` | 5.828 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q101_r1"]` | 6.061 μs (5%) | | 9.47 KiB (1%) | 202 | | `["circuitsim", "pftrajectories_union", "q1001_r1"]` | 23.213 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r100"]` | 132.997 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r10000"]` | 1.055 ms (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q101_r1"]` | 2.394 μs (5%) | | 96 bytes (1%) | 2 | | `["clifford", "dense", "cnot250_on_dense500_destab"]` | 11.491 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_dense500_stab"]` | 5.740 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_destab"]` | 1.121 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_stab"]` | 560.604 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot_on_dense500_destab"]` | 49.933 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_dense500_stab"]` | 24.456 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_destab"]` | 26.239 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_stab"]` | 13.735 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "dense500_on_dense500_destab"]` | 11.514 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_dense500_stab"]` | 5.744 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_destab"]` | 1.123 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_stab"]` | 560.611 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "symbolic", "cnot250_on_dense500_destab"]` | 1.432 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_dense500_stab"]` | 688.320 μs (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_destab"]` | 1.165 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_stab"]` | 570.632 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_destab"]` | 4.749 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_stab"]` | 2.304 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_destab"]` | 4.689 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_stab"]` | 2.294 μs (5%) | | | | | `["ecc", "evaluate_decoder", "shor_bp_comm"]` | 1.845 ms (5%) | | 1.78 MiB (1%) | 42707 | | `["ecc", "evaluate_decoder", "shor_bp_naivesyn"]` | 4.174 ms (5%) | | 3.61 MiB (1%) | 81648 | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 4.947 ms (5%) | | 3.77 MiB (1%) | 84162 | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 65.328 ms (5%) | | 4.76 MiB (1%) | 144207 | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 150.995 ms (5%) | | 9.57 MiB (1%) | 284018 | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 119.841 ms (5%) | | 9.66 MiB (1%) | 285104 | | `["ecc", "evaluate_decoder", "shor_table_comm"]` | 255.975 μs (5%) | | 241.42 KiB (1%) | 4593 | | `["ecc", "evaluate_decoder", "shor_table_naivesyn"]` | 863.045 μs (5%) | | 334.20 KiB (1%) | 4790 | | `["ecc", "evaluate_decoder", "shor_table_shorsyn"]` | 1.264 ms (5%) | | 427.73 KiB (1%) | 5876 | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 693.792 ms (5%) | 11.466 ms | 169.53 MiB (1%) | 1016786 | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 1.370 s (5%) | 24.615 ms | 343.32 MiB (1%) | 2080925 | | `["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]` | 1.418 s (5%) | 28.781 ms | 356.86 MiB (1%) | 2178180 | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 484.602 ms (5%) | | 8.11 MiB (1%) | 163696 | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 1.001 s (5%) | | 18.04 MiB (1%) | 360388 | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 953.327 ms (5%) | | 20.56 MiB (1%) | 392802 | | `["ecc", "evaluate_decoder", "toric8_pymatch_comm"]` | 4.271 ms (5%) | | 1.95 MiB (1%) | 23176 | | `["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]` | 12.474 ms (5%) | | 5.58 MiB (1%) | 79355 | | `["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]` | 21.033 ms (5%) | | 8.09 MiB (1%) | 111769 | | `["ecc", "evaluate_decoder", "toric8_table_comm"]` | 2.628 ms (5%) | | 1.64 MiB (1%) | 23082 | | `["ecc", "evaluate_decoder", "toric8_table_naivesyn"]` | 10.179 ms (5%) | | 4.93 MiB (1%) | 79160 | | `["ecc", "evaluate_decoder", "toric8_table_shorsyn"]` | 18.726 ms (5%) | | 7.44 MiB (1%) | 111574 | | `["pauli", "mul", "100"]` | 14.396 ns (5%) | | | | | `["pauli", "mul", "1000"]` | 18.209 ns (5%) | | | | | `["pauli", "mul", "100000"]` | 731.992 ns (5%) | | | | | `["pauli", "mul", "20000000"]` | 196.236 μs (5%) | | | | | `["stabilizer", "canon", "cano500"]` | 3.081 ms (5%) | | | | | `["stabilizer", "canon", "diag_cano500"]` | 676.397 μs (5%) | | | | | `["stabilizer", "canon", "diag_gott500"]` | 4.163 ms (5%) | | 5.19 MiB (1%) | 34521 | | `["stabilizer", "canon", "diag_rref500"]` | 665.978 μs (5%) | | | | | `["stabilizer", "canon", "gott500"]` | 6.575 ms (5%) | | 5.19 MiB (1%) | 34531 | | `["stabilizer", "canon", "md_cano500"]` | 1.543 ms (5%) | | | | | `["stabilizer", "canon", "md_rref500"]` | 1.556 ms (5%) | | | | | `["stabilizer", "canon", "rref500"]` | 3.142 ms (5%) | | | | | `["stabilizer", "project", "destabilizer"]` | 18.444 μs (5%) | | 288 bytes (1%) | 5 | | `["stabilizer", "project", "stabilizer"]` | 8.866 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "tensor", "diag_pow5_20"]` | 1.266 ms (5%) | | 23.97 MiB (1%) | 34 | | `["stabilizer", "tensor", "pow5_20"]` | 2.751 μs (5%) | | 6.44 KiB (1%) | 31 | | `["stabilizer", "trace", "destabilizer"]` | 31.439 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "trace", "stabilizer"]` | 24.205 μs (5%) | | 112 bytes (1%) | 3 | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 2528 s 0 s 261 s 10527 s 0 s #2 0 MHz 3405 s 0 s 265 s 9654 s 0 s #3 0 MHz 3625 s 0 s 267 s 9440 s 0 s #4 0 MHz 3969 s 0 s 264 s 9093 s 0 s Memory: 15.606494903564453 GB (13249.9453125 MB free) Uptime: 1338.58 sec Load Avg: 1.15 1.12 0.92 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Runtime information | Runtime Info | | |:--|:--| | BLAS #threads | 2 | | `BLAS.vendor()` | `lbt` | | `Sys.CPU_THREADS` | 4 | `lscpu` output: Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 48 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Vendor ID: AuthenticAMD Model name: AMD EPYC 7763 64-Core Processor CPU family: 25 Model: 1 Thread(s) per core: 2 Core(s) per socket: 2 Socket(s): 1 Stepping: 1 BogoMIPS: 4890.86 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm Virtualization: AMD-V Hypervisor vendor: Microsoft Virtualization type: full L1d cache: 64 KiB (2 instances) L1i cache: 64 KiB (2 instances) L2 cache: 1 MiB (2 instances) L3 cache: 32 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-3 Vulnerability Gather data sampling: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode Vulnerability Spec store bypass: Vulnerable Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected | Cpu Property | Value | |:------------------ |:---------------------------------------------------------- | | Brand | AMD EPYC 7763 64-Core Processor | | Vendor | :AMD | | Architecture | :Unknown | | Model | Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00 | | Cores | 16 physical cores, 16 logical cores (on executing CPU) | | | No Hyperthreading hardware capability detected | | Clock Frequencies | Not supported by CPU | | Data Cache | Level 1:3 : (32, 512, 32768) kbytes | | | 64 byte cache line size | | Address Size | 48 bits virtual, 48 bits physical | | SIMD | 256 bit = 32 byte max. SIMD vector size | | Time Stamp Counter | TSC is accessible via `rdtsc` | | | TSC runs at constant rate (invariant from clock frequency) | | Perf. Monitoring | Performance Monitoring Counters (PMC) are not supported | | Hypervisor | Yes, Microsoft |
Benchmark Result
# Judge result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmarks: - Target: 18 Apr 2024 - 15:45 - Baseline: 18 Apr 2024 - 15:52 * Package commits: - Target: d01f41 - Baseline: 64f695 * Julia commits: - Target: 0f7674 - Baseline: 0f7674 * Julia command flags: - Target: None - Baseline: None * Environment variables: - Target: None - Baseline: None ## Results A ratio greater than `1.0` denotes a possible regression (marked with :x:), while a ratio less than `1.0` denotes a possible improvement (marked with :white_check_mark:). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds). | ID | time ratio | memory ratio | |--------------------------------------------------------------------|------------------------------|------------------------------| | `["clifford", "dense", "cnot_on_dense500_stab"]` | 1.06 (5%) :x: | 1.00 (1%) | | `["clifford", "dense", "dense500_on_diag500_destab"]` | 0.87 (5%) :white_check_mark: | 1.00 (1%) | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 0.92 (5%) :white_check_mark: | 0.98 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 0.34 (5%) :white_check_mark: | 0.83 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 0.33 (5%) :white_check_mark: | 0.83 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 0.38 (5%) :white_check_mark: | 0.83 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 1.01 (5%) | 1.02 (1%) :x: | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 0.97 (5%) | 0.97 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 0.80 (5%) :white_check_mark: | 0.90 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 0.83 (5%) :white_check_mark: | 0.91 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 0.83 (5%) :white_check_mark: | 0.92 (1%) :white_check_mark: | | `["pauli", "mul", "1000"]` | 1.07 (5%) :x: | 1.00 (1%) | | `["stabilizer", "tensor", "diag_pow5_20"]` | 0.91 (5%) :white_check_mark: | 1.00 (1%) | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ### Target ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 2081 s 0 s 169 s 6834 s 0 s #2 0 MHz 2125 s 0 s 185 s 6784 s 0 s #3 0 MHz 2143 s 0 s 191 s 6744 s 0 s #4 0 MHz 2729 s 0 s 192 s 6162 s 0 s Memory: 15.606494903564453 GB (13380.7578125 MB free) Uptime: 913.16 sec Load Avg: 1.03 1.01 0.71 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` ### Baseline ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 3162 s 0 s 269 s 9933 s 0 s #2 0 MHz 3507 s 0 s 282 s 9586 s 0 s #3 0 MHz 2978 s 0 s 281 s 10099 s 0 s #4 0 MHz 3861 s 0 s 295 s 9207 s 0 s Memory: 15.606494903564453 GB (13273.22265625 MB free) Uptime: 1342.65 sec Load Avg: 1.16 1.09 0.88 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Target result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmark: 18 Apr 2024 - 15:45 * Package commit: d01f41 * Julia commit: 0f7674 * Julia command flags: None * Environment variables: None ## Results Below is a table of this job's results, obtained by running the benchmarks. The values listed in the `ID` column have the structure `[parent_group, child_group, ..., key]`, and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero. | ID | time | GC time | memory | allocations | |--------------------------------------------------------------------|----------------:|----------:|----------------:|------------:| | `["circuitsim", "mctrajectories", "q1001_r1"]` | 15.191 ms (5%) | | 500.77 KiB (1%) | 18019 | | `["circuitsim", "mctrajectories", "q101_r1"]` | 172.271 μs (5%) | | 50.53 KiB (1%) | 1818 | | `["circuitsim", "mctrajectories_sumtype", "q1001_r1"]` | 13.316 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_sumtype", "q101_r1"]` | 119.092 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "mctrajectories_union", "q1001_r1"]` | 13.246 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_union", "q101_r1"]` | 117.910 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "pftrajectories", "q1001_r1"]` | 59.982 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r100"]` | 177.401 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r10000"]` | 1.146 ms (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q101_r1"]` | 6.031 μs (5%) | | 9.42 KiB (1%) | 201 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r1"]` | 44.583 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r100"]` | 158.977 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]` | 1.129 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]` | 5.872 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q101_r1"]` | 4.448 μs (5%) | | 9.47 KiB (1%) | 202 | | `["circuitsim", "pftrajectories_union", "q1001_r1"]` | 22.913 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r100"]` | 136.535 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r10000"]` | 1.098 ms (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q101_r1"]` | 2.364 μs (5%) | | 96 bytes (1%) | 2 | | `["clifford", "dense", "cnot250_on_dense500_destab"]` | 11.658 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_dense500_stab"]` | 5.849 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_destab"]` | 1.123 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_stab"]` | 560.655 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot_on_dense500_destab"]` | 51.215 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_dense500_stab"]` | 24.195 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_destab"]` | 26.329 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_stab"]` | 13.616 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "dense500_on_dense500_destab"]` | 11.653 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_dense500_stab"]` | 5.855 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_destab"]` | 980.047 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_stab"]` | 560.655 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "symbolic", "cnot250_on_dense500_destab"]` | 1.440 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_dense500_stab"]` | 686.199 μs (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_destab"]` | 1.160 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_stab"]` | 569.271 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_destab"]` | 4.739 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_stab"]` | 2.314 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_destab"]` | 4.658 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_stab"]` | 2.284 μs (5%) | | | | | `["ecc", "evaluate_decoder", "shor_bp_comm"]` | 1.820 ms (5%) | | 1.77 MiB (1%) | 42665 | | `["ecc", "evaluate_decoder", "shor_bp_naivesyn"]` | 4.169 ms (5%) | | 3.61 MiB (1%) | 81704 | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 4.548 ms (5%) | | 3.71 MiB (1%) | 82902 | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 24.183 ms (5%) | | 3.97 MiB (1%) | 116082 | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 48.685 ms (5%) | | 7.97 MiB (1%) | 227768 | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 49.012 ms (5%) | | 8.06 MiB (1%) | 228854 | | `["ecc", "evaluate_decoder", "shor_table_comm"]` | 255.006 μs (5%) | | 241.42 KiB (1%) | 4593 | | `["ecc", "evaluate_decoder", "shor_table_naivesyn"]` | 870.613 μs (5%) | | 334.20 KiB (1%) | 4790 | | `["ecc", "evaluate_decoder", "shor_table_shorsyn"]` | 1.264 ms (5%) | | 427.73 KiB (1%) | 5876 | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 676.475 ms (5%) | 13.812 ms | 170.14 MiB (1%) | 1020398 | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 1.377 s (5%) | 27.679 ms | 350.41 MiB (1%) | 2122596 | | `["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]` | 1.382 s (5%) | 29.151 ms | 351.42 MiB (1%) | 2146183 | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 397.337 ms (5%) | | 7.31 MiB (1%) | 135571 | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 809.669 ms (5%) | | 16.45 MiB (1%) | 304138 | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 825.844 ms (5%) | | 18.96 MiB (1%) | 336552 | | `["ecc", "evaluate_decoder", "toric8_pymatch_comm"]` | 4.207 ms (5%) | | 1.95 MiB (1%) | 23176 | | `["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]` | 12.550 ms (5%) | | 5.58 MiB (1%) | 79355 | | `["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]` | 20.928 ms (5%) | | 8.09 MiB (1%) | 111769 | | `["ecc", "evaluate_decoder", "toric8_table_comm"]` | 2.634 ms (5%) | | 1.64 MiB (1%) | 23082 | | `["ecc", "evaluate_decoder", "toric8_table_naivesyn"]` | 10.279 ms (5%) | | 4.93 MiB (1%) | 79160 | | `["ecc", "evaluate_decoder", "toric8_table_shorsyn"]` | 18.603 ms (5%) | | 7.44 MiB (1%) | 111574 | | `["pauli", "mul", "100"]` | 14.827 ns (5%) | | | | | `["pauli", "mul", "1000"]` | 18.831 ns (5%) | | | | | `["pauli", "mul", "100000"]` | 731.835 ns (5%) | | | | | `["pauli", "mul", "20000000"]` | 182.841 μs (5%) | | | | | `["stabilizer", "canon", "cano500"]` | 3.046 ms (5%) | | | | | `["stabilizer", "canon", "diag_cano500"]` | 708.661 μs (5%) | | | | | `["stabilizer", "canon", "diag_gott500"]` | 4.037 ms (5%) | | 5.19 MiB (1%) | 34521 | | `["stabilizer", "canon", "diag_rref500"]` | 704.153 μs (5%) | | | | | `["stabilizer", "canon", "gott500"]` | 6.382 ms (5%) | | 5.19 MiB (1%) | 34531 | | `["stabilizer", "canon", "md_cano500"]` | 1.544 ms (5%) | | | | | `["stabilizer", "canon", "md_rref500"]` | 1.574 ms (5%) | | | | | `["stabilizer", "canon", "rref500"]` | 3.095 ms (5%) | | | | | `["stabilizer", "project", "destabilizer"]` | 18.374 μs (5%) | | 288 bytes (1%) | 5 | | `["stabilizer", "project", "stabilizer"]` | 9.468 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "tensor", "diag_pow5_20"]` | 1.091 ms (5%) | | 23.97 MiB (1%) | 34 | | `["stabilizer", "tensor", "pow5_20"]` | 2.809 μs (5%) | | 6.44 KiB (1%) | 31 | | `["stabilizer", "trace", "destabilizer"]` | 31.349 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "trace", "stabilizer"]` | 24.255 μs (5%) | | 112 bytes (1%) | 3 | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - ...*[Comment body truncated]*
Benchmark Result
# Judge result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmarks: - Target: 18 Apr 2024 - 15:07 - Baseline: 18 Apr 2024 - 15:14 * Package commits: - Target: 65fcd1 - Baseline: 64f695 * Julia commits: - Target: 0f7674 - Baseline: 0f7674 * Julia command flags: - Target: None - Baseline: None * Environment variables: - Target: None - Baseline: None ## Results A ratio greater than `1.0` denotes a possible regression (marked with :x:), while a ratio less than `1.0` denotes a possible improvement (marked with :white_check_mark:). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds). | ID | time ratio | memory ratio | |--------------------------------------------------------------------|------------------------------|------------------------------| | `["circuitsim", "mctrajectories_sumtype", "q1001_r1"]` | 1.07 (5%) :x: | 1.00 (1%) | | `["circuitsim", "mctrajectories_union", "q1001_r1"]` | 1.06 (5%) :x: | 1.00 (1%) | | `["circuitsim", "pftrajectories", "q101_r1"]` | 0.95 (5%) :white_check_mark: | 1.00 (1%) | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 0.14 (5%) :white_check_mark: | 0.98 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 0.44 (5%) :white_check_mark: | 0.83 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 0.35 (5%) :white_check_mark: | 0.83 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 0.43 (5%) :white_check_mark: | 0.83 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 0.98 (5%) | 0.97 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 0.99 (5%) | 0.98 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 0.86 (5%) :white_check_mark: | 0.90 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 0.84 (5%) :white_check_mark: | 0.91 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 0.87 (5%) :white_check_mark: | 0.92 (1%) :white_check_mark: | | `["pauli", "mul", "20000000"]` | 1.07 (5%) :x: | 1.00 (1%) | | `["stabilizer", "tensor", "diag_pow5_20"]` | 1.07 (5%) :x: | 1.00 (1%) | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ### Target ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 2467 s 0 s 187 s 6679 s 0 s #2 0 MHz 2552 s 0 s 159 s 6616 s 0 s #3 0 MHz 2576 s 0 s 188 s 6575 s 0 s #4 0 MHz 1738 s 0 s 195 s 7404 s 0 s Memory: 15.606494903564453 GB (13354.48046875 MB free) Uptime: 937.94 sec Load Avg: 1.05 1.06 0.78 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` ### Baseline ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 3109 s 0 s 286 s 10132 s 0 s #2 0 MHz 3373 s 0 s 241 s 9907 s 0 s #3 0 MHz 3648 s 0 s 276 s 9611 s 0 s #4 0 MHz 3504 s 0 s 294 s 9736 s 0 s Memory: 15.606494903564453 GB (13194.15625 MB free) Uptime: 1358.95 sec Load Avg: 1.03 1.09 0.93 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Target result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmark: 18 Apr 2024 - 15:7 * Package commit: 65fcd1 * Julia commit: 0f7674 * Julia command flags: None * Environment variables: None ## Results Below is a table of this job's results, obtained by running the benchmarks. The values listed in the `ID` column have the structure `[parent_group, child_group, ..., key]`, and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero. | ID | time | GC time | memory | allocations | |--------------------------------------------------------------------|----------------:|----------:|----------------:|------------:| | `["circuitsim", "mctrajectories", "q1001_r1"]` | 15.667 ms (5%) | | 500.77 KiB (1%) | 18019 | | `["circuitsim", "mctrajectories", "q101_r1"]` | 170.619 μs (5%) | | 50.53 KiB (1%) | 1818 | | `["circuitsim", "mctrajectories_sumtype", "q1001_r1"]` | 14.108 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_sumtype", "q101_r1"]` | 117.570 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "mctrajectories_union", "q1001_r1"]` | 13.851 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_union", "q101_r1"]` | 116.217 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "pftrajectories", "q1001_r1"]` | 64.560 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r100"]` | 177.432 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r10000"]` | 1.100 ms (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q101_r1"]` | 6.473 μs (5%) | | 9.42 KiB (1%) | 201 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r1"]` | 44.173 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r100"]` | 155.281 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]` | 1.086 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]` | 5.838 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q101_r1"]` | 4.518 μs (5%) | | 9.47 KiB (1%) | 202 | | `["circuitsim", "pftrajectories_union", "q1001_r1"]` | 22.612 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r100"]` | 131.627 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r10000"]` | 1.053 ms (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q101_r1"]` | 2.344 μs (5%) | | 96 bytes (1%) | 2 | | `["clifford", "dense", "cnot250_on_dense500_destab"]` | 11.594 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_dense500_stab"]` | 5.801 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_destab"]` | 1.123 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_stab"]` | 490.437 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot_on_dense500_destab"]` | 50.916 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_dense500_stab"]` | 23.183 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_destab"]` | 26.309 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_stab"]` | 13.365 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "dense500_on_dense500_destab"]` | 11.568 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_dense500_stab"]` | 5.805 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_destab"]` | 980.163 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_stab"]` | 490.277 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "symbolic", "cnot250_on_dense500_destab"]` | 1.440 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_dense500_stab"]` | 688.117 μs (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_destab"]` | 1.160 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_stab"]` | 569.645 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_destab"]` | 4.749 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_stab"]` | 2.304 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_destab"]` | 4.688 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_stab"]` | 2.294 μs (5%) | | | | | `["ecc", "evaluate_decoder", "shor_bp_comm"]` | 1.859 ms (5%) | | 1.77 MiB (1%) | 42581 | | `["ecc", "evaluate_decoder", "shor_bp_naivesyn"]` | 4.165 ms (5%) | | 3.61 MiB (1%) | 81592 | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 4.566 ms (5%) | | 3.71 MiB (1%) | 82902 | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 25.345 ms (5%) | | 3.97 MiB (1%) | 116082 | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 50.255 ms (5%) | | 7.97 MiB (1%) | 227768 | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 51.436 ms (5%) | | 8.06 MiB (1%) | 228854 | | `["ecc", "evaluate_decoder", "shor_table_comm"]` | 259.135 μs (5%) | | 241.42 KiB (1%) | 4593 | | `["ecc", "evaluate_decoder", "shor_table_naivesyn"]` | 868.234 μs (5%) | | 334.20 KiB (1%) | 4790 | | `["ecc", "evaluate_decoder", "shor_table_shorsyn"]` | 1.268 ms (5%) | | 427.73 KiB (1%) | 5876 | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 688.833 ms (5%) | 20.256 ms | 169.85 MiB (1%) | 1018690 | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 1.409 s (5%) | 41.903 ms | 347.87 MiB (1%) | 2107672 | | `["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]` | 1.410 s (5%) | 39.383 ms | 348.54 MiB (1%) | 2129271 | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 401.165 ms (5%) | | 7.31 MiB (1%) | 135571 | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 825.480 ms (5%) | | 16.45 MiB (1%) | 304138 | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 842.126 ms (5%) | | 18.96 MiB (1%) | 336552 | | `["ecc", "evaluate_decoder", "toric8_pymatch_comm"]` | 4.250 ms (5%) | | 1.95 MiB (1%) | 23176 | | `["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]` | 12.670 ms (5%) | | 5.58 MiB (1%) | 79355 | | `["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]` | 21.257 ms (5%) | | 8.09 MiB (1%) | 111769 | | `["ecc", "evaluate_decoder", "toric8_table_comm"]` | 2.651 ms (5%) | | 1.64 MiB (1%) | 23082 | | `["ecc", "evaluate_decoder", "toric8_table_naivesyn"]` | 10.185 ms (5%) | | 4.93 MiB (1%) | 79160 | | `["ecc", "evaluate_decoder", "toric8_table_shorsyn"]` | 18.745 ms (5%) | | 7.44 MiB (1%) | 111574 | | `["pauli", "mul", "100"]` | 14.415 ns (5%) | | | | | `["pauli", "mul", "1000"]` | 18.218 ns (5%) | | | | | `["pauli", "mul", "100000"]` | 722.244 ns (5%) | | | | | `["pauli", "mul", "20000000"]` | 222.988 μs (5%) | | | | | `["stabilizer", "canon", "cano500"]` | 3.081 ms (5%) | | | | | `["stabilizer", "canon", "diag_cano500"]` | 676.466 μs (5%) | | | | | `["stabilizer", "canon", "diag_gott500"]` | 4.147 ms (5%) | | 5.19 MiB (1%) | 34521 | | `["stabilizer", "canon", "diag_rref500"]` | 665.755 μs (5%) | | | | | `["stabilizer", "canon", "gott500"]` | 6.606 ms (5%) | | 5.19 MiB (1%) | 34531 | | `["stabilizer", "canon", "md_cano500"]` | 1.592 ms (5%) | | | | | `["stabilizer", "canon", "md_rref500"]` | 1.632 ms (5%) | | | | | `["stabilizer", "canon", "rref500"]` | 3.144 ms (5%) | | | | | `["stabilizer", "project", "destabilizer"]` | 18.214 μs (5%) | | 288 bytes (1%) | 5 | | `["stabilizer", "project", "stabilizer"]` | 9.067 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "tensor", "diag_pow5_20"]` | 1.527 ms (5%) | | 23.97 MiB (1%) | 34 | | `["stabilizer", "tensor", "pow5_20"]` | 2.836 μs (5%) | | 6.44 KiB (1%) | 31 | | `["stabilizer", "trace", "destabilizer"]` | 34.424 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "trace", "stabilizer"]` | 24.175 μs (5%) | | 112 bytes (1%) | 3 | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 2467 s 0 s 187 s 6679 s 0 s #2 0 MHz 2552 s 0 s 159 s 6616 s 0 s #3 0 MHz 2576 s 0 s 188 s 6575 s 0 s #4 0 MHz 1738 s 0 s 195 s 7404 s 0 s Memory: 15.606494903564453 GB (13354.48046875 MB free) Uptime: 937.94 sec Load Avg: 1.05 1.06 0.78 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Baseline result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmark: 18 Apr 2024 - 15:14 * Package commit: 64f695 * Julia commit: 0f7674 * Julia command flags: None * Environment variables: None ## Results Below is a table of this job's results, obtained by running the benchmarks. The values listed in the `ID` column have the structure `[parent_group, child_group, ..., key]`, and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero. | ID | time | GC time | memory | allocations | |--------------------------------------------------------------------|----------------:|----------:|----------------:|------------:| | `["circuitsim", "mctrajectories", "q1001_r1"]` | 14.983 ms (5%) | | 500.77 KiB (1%) | 18019 | | `["circuitsim", "mctrajectories", "q101_r1"]` | 171.541 μs (5%) | | 50.53 KiB (1%) | 1818 | | `["circuitsim", "mctrajectories_sumtype", "q1001_r1"]` | 13.181 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_sumtype", "q101_r1"]` | 118.942 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "mctrajectories_union", "q1001_r1"]` | 13.107 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_union", "q101_r1"]` | 117.891 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "pftrajectories", "q1001_r1"]` | 67.395 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r100"]` | 175.468 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r10000"]` | 1.145 ms (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q101_r1"]` | 6.833 μs (5%) | | 9.42 KiB (1%) | 201 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r1"]` | 44.704 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r100"]` | 159.598 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]` | 1.135 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]` | 5.820 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q101_r1"]` | 4.488 μs (5%) | | 9.47 KiB (1%) | 202 | | `["circuitsim", "pftrajectories_union", "q1001_r1"]` | 22.602 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r100"]` | 136.385 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r10000"]` | 1.102 ms (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q101_r1"]` | 2.344 μs (5%) | | 96 bytes (1%) | 2 | | `["clifford", "dense", "cnot250_on_dense500_destab"]` | 11.639 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_dense500_stab"]` | 5.793 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_destab"]` | 1.124 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_stab"]` | 490.456 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot_on_dense500_destab"]` | 51.136 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_dense500_stab"]` | 23.404 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_destab"]` | 26.690 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_stab"]` | 13.374 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "dense500_on_dense500_destab"]` | 11.609 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_dense500_stab"]` | 5.766 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_destab"]` | 980.322 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_stab"]` | 490.237 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "symbolic", "cnot250_on_dense500_destab"]` | 1.441 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_dense500_stab"]` | 687.545 μs (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_destab"]` | 1.161 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_stab"]` | 570.315 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_destab"]` | 4.728 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_stab"]` | 2.304 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_destab"]` | 4.659 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_stab"]` | 2.314 μs (5%) | | | | | `["ecc", "evaluate_decoder", "shor_bp_comm"]` | 1.847 ms (5%) | | 1.77 MiB (1%) | 42581 | | `["ecc", "evaluate_decoder", "shor_bp_naivesyn"]` | 4.165 ms (5%) | | 3.61 MiB (1%) | 81648 | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 32.173 ms (5%) | 26.267 ms | 3.79 MiB (1%) | 84708 | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 57.151 ms (5%) | | 4.76 MiB (1%) | 144207 | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 145.566 ms (5%) | | 9.57 MiB (1%) | 284018 | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 118.722 ms (5%) | | 9.66 MiB (1%) | 285104 | | `["ecc", "evaluate_decoder", "shor_table_comm"]` | 254.355 μs (5%) | | 241.42 KiB (1%) | 4593 | | `["ecc", "evaluate_decoder", "shor_table_naivesyn"]` | 877.901 μs (5%) | | 334.20 KiB (1%) | 4790 | | `["ecc", "evaluate_decoder", "shor_table_shorsyn"]` | 1.278 ms (5%) | | 427.73 KiB (1%) | 5876 | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 703.965 ms (5%) | 15.404 ms | 175.59 MiB (1%) | 1052444 | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 1.420 s (5%) | 30.498 ms | 355.25 MiB (1%) | 2151086 | | `["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]` | 1.410 s (5%) | 36.070 ms | 350.22 MiB (1%) | 2139113 | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 464.203 ms (5%) | | 8.11 MiB (1%) | 163696 | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 978.670 ms (5%) | | 18.04 MiB (1%) | 360388 | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 963.926 ms (5%) | | 20.56 MiB (1%) | 392802 | | `["ecc", "evaluate_decoder", "toric8_pymatch_comm"]` | 4.267 ms (5%) | | 1.95 MiB (1%) | 23176 | | `["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]` | 12.732 ms (5%) | | 5.58 MiB (1%) | 79355 | | `["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]` | 21.370 ms (5%) | | 8.09 MiB (1%) | 111769 | | `["ecc", "evaluate_decoder", "toric8_table_comm"]` | 2.639 ms (5%) | | 1.64 MiB (1%) | 23082 | | `["ecc", "evaluate_decoder", "toric8_table_naivesyn"]` | 10.456 ms (5%) | | 4.93 MiB (1%) | 79160 | | `["ecc", "evaluate_decoder", "toric8_table_shorsyn"]` | 19.077 ms (5%) | | 7.44 MiB (1%) | 111574 | | `["pauli", "mul", "100"]` | 14.505 ns (5%) | | | | | `["pauli", "mul", "1000"]` | 18.820 ns (5%) | | | | | `["pauli", "mul", "100000"]` | 732.911 ns (5%) | | | | | `["pauli", "mul", "20000000"]` | 209.321 μs (5%) | | | | | `["stabilizer", "canon", "cano500"]` | 3.081 ms (5%) | | | | | `["stabilizer", "canon", "diag_cano500"]` | 676.744 μs (5%) | | | | | `["stabilizer", "canon", "diag_gott500"]` | 4.131 ms (5%) | | 5.19 MiB (1%) | 34521 | | `["stabilizer", "canon", "diag_rref500"]` | 665.774 μs (5%) | | | | | `["stabilizer", "canon", "gott500"]` | 6.521 ms (5%) | | 5.19 MiB (1%) | 34531 | | `["stabilizer", "canon", "md_cano500"]` | 1.606 ms (5%) | | | | | `["stabilizer", "canon", "md_rref500"]` | 1.649 ms (5%) | | | | | `["stabilizer", "canon", "rref500"]` | 3.147 ms (5%) | | | | | `["stabilizer", "project", "destabilizer"]` | 18.425 μs (5%) | | 288 bytes (1%) | 5 | | `["stabilizer", "project", "stabilizer"]` | 9.217 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "tensor", "diag_pow5_20"]` | 1.422 ms (5%) | | 23.97 MiB (1%) | 34 | | `["stabilizer", "tensor", "pow5_20"]` | 2.719 μs (5%) | | 6.44 KiB (1%) | 31 | | `["stabilizer", "trace", "destabilizer"]` | 34.304 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "trace", "stabilizer"]` | 24.135 μs (5%) | | 112 bytes (1%) | 3 | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 3109 s 0 s 286 s 10132 s 0 s #2 0 MHz 3373 s 0 s 241 s 9907 s 0 s #3 0 MHz 3648 s 0 s 276 s 9611 s 0 s #4 0 MHz 3504 s 0 s 294 s 9736 s 0 s Memory: 15.606494903564453 GB (13194.15625 MB free) Uptime: 1358.95 sec Load Avg: 1.03 1.09 0.93 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Runtime information | Runtime Info | | |:--|:--| | BLAS #threads | 2 | | `BLAS.vendor()` | `lbt` | | `Sys.CPU_THREADS` | 4 | `lscpu` output: Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 48 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Vendor ID: AuthenticAMD Model name: AMD EPYC 7763 64-Core Processor CPU family: 25 Model: 1 Thread(s) per core: 2 Core(s) per socket: 2 Socket(s): 1 Stepping: 1 BogoMIPS: 4890.86 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm Virtualization: AMD-V Hypervisor vendor: Microsoft Virtualization type: full L1d cache: 64 KiB (2 instances) L1i cache: 64 KiB (2 instances) L2 cache: 1 MiB (2 instances) L3 cache: 32 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-3 Vulnerability Gather data sampling: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode Vulnerability Spec store bypass: Vulnerable Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected | Cpu Property | Value | |:------------------ |:---------------------------------------------------------- | | Brand | AMD EPYC 7763 64-Core Processor | | Vendor | :AMD | | Architecture | :Unknown | | Model | Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00 | | Cores | 16 physical cores, 16 logical cores (on executing CPU) | | | No Hyperthreading hardware capability detected | | Clock Frequencies | Not supported by CPU | | Data Cache | Level 1:3 : (32, 512, 32768) kbytes | | | 64 byte cache line size | | Address Size | 48 bits virtual, 48 bits physical | | SIMD | 256 bit = 32 byte max. SIMD vector size | | Time Stamp Counter | TSC is accessible via `rdtsc` | | | TSC runs at constant rate (invariant from clock frequency) | | Perf. Monitoring | Performance Monitoring Counters (PMC) are not supported | | Hypervisor | Yes, Microsoft |Benchmark Result
# Judge result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmarks: - Target: 18 Apr 2024 - 15:15 - Baseline: 18 Apr 2024 - 15:22 * Package commits: - Target: be8388 - Baseline: 64f695 * Julia commits: - Target: 0f7674 - Baseline: 0f7674 * Julia command flags: - Target: None - Baseline: None * Environment variables: - Target: None - Baseline: None ## Results A ratio greater than `1.0` denotes a possible regression (marked with :x:), while a ratio less than `1.0` denotes a possible improvement (marked with :white_check_mark:). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds). | ID | time ratio | memory ratio | |--------------------------------------------------------------------|------------------------------|------------------------------| | `["circuitsim", "pftrajectories", "q1001_r1"]` | 0.95 (5%) :white_check_mark: | 1.00 (1%) | | `["circuitsim", "pftrajectories", "q101_r1"]` | 0.95 (5%) :white_check_mark: | 1.00 (1%) | | `["circuitsim", "pftrajectories_sumtype", "q1001_r1"]` | 0.73 (5%) :white_check_mark: | 1.00 (1%) | | `["circuitsim", "pftrajectories_sumtype", "q1001_r100"]` | 0.94 (5%) :white_check_mark: | 1.00 (1%) | | `["circuitsim", "pftrajectories_sumtype", "q101_r1"]` | 0.73 (5%) :white_check_mark: | 1.00 (1%) | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 0.92 (5%) :white_check_mark: | 0.98 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 0.38 (5%) :white_check_mark: | 0.83 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 0.33 (5%) :white_check_mark: | 0.83 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 0.42 (5%) :white_check_mark: | 0.83 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 0.97 (5%) | 1.02 (1%) :x: | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 1.04 (5%) | 1.02 (1%) :x: | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 0.80 (5%) :white_check_mark: | 0.90 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 0.81 (5%) :white_check_mark: | 0.91 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 0.85 (5%) :white_check_mark: | 0.92 (1%) :white_check_mark: | | `["stabilizer", "trace", "destabilizer"]` | 1.06 (5%) :x: | 1.00 (1%) | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ### Target ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 1779 s 0 s 162 s 7262 s 0 s #2 0 MHz 2054 s 0 s 172 s 6979 s 0 s #3 0 MHz 2571 s 0 s 188 s 6450 s 0 s #4 0 MHz 2907 s 0 s 186 s 6108 s 0 s Memory: 15.606494903564453 GB (13357.609375 MB free) Uptime: 925.16 sec Load Avg: 1.14 1.07 0.76 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` ### Baseline ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 2528 s 0 s 261 s 10527 s 0 s #2 0 MHz 3405 s 0 s 265 s 9654 s 0 s #3 0 MHz 3625 s 0 s 267 s 9440 s 0 s #4 0 MHz 3969 s 0 s 264 s 9093 s 0 s Memory: 15.606494903564453 GB (13249.9453125 MB free) Uptime: 1338.58 sec Load Avg: 1.15 1.12 0.92 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Target result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmark: 18 Apr 2024 - 15:15 * Package commit: be8388 * Julia commit: 0f7674 * Julia command flags: None * Environment variables: None ## Results Below is a table of this job's results, obtained by running the benchmarks. The values listed in the `ID` column have the structure `[parent_group, child_group, ..., key]`, and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero. | ID | time | GC time | memory | allocations | |--------------------------------------------------------------------|----------------:|----------:|----------------:|------------:| | `["circuitsim", "mctrajectories", "q1001_r1"]` | 15.111 ms (5%) | | 500.77 KiB (1%) | 18019 | | `["circuitsim", "mctrajectories", "q101_r1"]` | 169.545 μs (5%) | | 50.53 KiB (1%) | 1818 | | `["circuitsim", "mctrajectories_sumtype", "q1001_r1"]` | 12.961 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_sumtype", "q101_r1"]` | 118.391 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "mctrajectories_union", "q1001_r1"]` | 13.136 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_union", "q101_r1"]` | 117.288 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "pftrajectories", "q1001_r1"]` | 59.882 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r100"]` | 180.045 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r10000"]` | 1.148 ms (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q101_r1"]` | 6.031 μs (5%) | | 9.42 KiB (1%) | 201 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r1"]` | 44.072 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r100"]` | 158.475 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]` | 1.130 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]` | 5.821 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q101_r1"]` | 4.448 μs (5%) | | 9.47 KiB (1%) | 202 | | `["circuitsim", "pftrajectories_union", "q1001_r1"]` | 22.611 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r100"]` | 136.694 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r10000"]` | 1.099 ms (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q101_r1"]` | 2.344 μs (5%) | | 96 bytes (1%) | 2 | | `["clifford", "dense", "cnot250_on_dense500_destab"]` | 11.496 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_dense500_stab"]` | 5.734 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_destab"]` | 1.121 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_stab"]` | 560.591 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot_on_dense500_destab"]` | 50.153 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_dense500_stab"]` | 24.395 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_destab"]` | 26.219 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_stab"]` | 13.755 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "dense500_on_dense500_destab"]` | 11.509 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_dense500_stab"]` | 5.740 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_destab"]` | 1.123 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_stab"]` | 560.612 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "symbolic", "cnot250_on_dense500_destab"]` | 1.446 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_dense500_stab"]` | 687.718 μs (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_destab"]` | 1.158 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_stab"]` | 569.930 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_destab"]` | 4.739 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_stab"]` | 2.304 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_destab"]` | 4.708 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_stab"]` | 2.304 μs (5%) | | | | | `["ecc", "evaluate_decoder", "shor_bp_comm"]` | 1.812 ms (5%) | | 1.77 MiB (1%) | 42609 | | `["ecc", "evaluate_decoder", "shor_bp_naivesyn"]` | 4.123 ms (5%) | | 3.62 MiB (1%) | 81648 | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 4.539 ms (5%) | | 3.70 MiB (1%) | 82636 | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 24.597 ms (5%) | | 3.97 MiB (1%) | 116082 | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 49.884 ms (5%) | | 7.97 MiB (1%) | 227768 | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 50.023 ms (5%) | | 8.06 MiB (1%) | 228854 | | `["ecc", "evaluate_decoder", "shor_table_comm"]` | 254.523 μs (5%) | | 241.42 KiB (1%) | 4593 | | `["ecc", "evaluate_decoder", "shor_table_naivesyn"]` | 864.997 μs (5%) | | 334.20 KiB (1%) | 4790 | | `["ecc", "evaluate_decoder", "shor_table_shorsyn"]` | 1.265 ms (5%) | | 427.73 KiB (1%) | 5876 | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 675.963 ms (5%) | 17.018 ms | 172.23 MiB (1%) | 1032711 | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 1.423 s (5%) | 32.888 ms | 350.38 MiB (1%) | 2122463 | | `["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]` | 1.443 s (5%) | 35.364 ms | 356.70 MiB (1%) | 2177228 | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 386.739 ms (5%) | | 7.31 MiB (1%) | 135571 | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 805.859 ms (5%) | | 16.45 MiB (1%) | 304138 | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 811.335 ms (5%) | | 18.96 MiB (1%) | 336552 | | `["ecc", "evaluate_decoder", "toric8_pymatch_comm"]` | 4.273 ms (5%) | | 1.95 MiB (1%) | 23176 | | `["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]` | 12.547 ms (5%) | | 5.58 MiB (1%) | 79355 | | `["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]` | 21.021 ms (5%) | | 8.09 MiB (1%) | 111769 | | `["ecc", "evaluate_decoder", "toric8_table_comm"]` | 2.616 ms (5%) | | 1.64 MiB (1%) | 23082 | | `["ecc", "evaluate_decoder", "toric8_table_naivesyn"]` | 10.164 ms (5%) | | 4.93 MiB (1%) | 79160 | | `["ecc", "evaluate_decoder", "toric8_table_shorsyn"]` | 18.616 ms (5%) | | 7.44 MiB (1%) | 111574 | | `["pauli", "mul", "100"]` | 14.395 ns (5%) | | | | | `["pauli", "mul", "1000"]` | 17.918 ns (5%) | | | | | `["pauli", "mul", "100000"]` | 722.756 ns (5%) | | | | | `["pauli", "mul", "20000000"]` | 192.749 μs (5%) | | | | | `["stabilizer", "canon", "cano500"]` | 3.082 ms (5%) | | | | | `["stabilizer", "canon", "diag_cano500"]` | 676.368 μs (5%) | | | | | `["stabilizer", "canon", "diag_gott500"]` | 4.178 ms (5%) | | 5.19 MiB (1%) | 34521 | | `["stabilizer", "canon", "diag_rref500"]` | 665.897 μs (5%) | | | | | `["stabilizer", "canon", "gott500"]` | 6.580 ms (5%) | | 5.19 MiB (1%) | 34531 | | `["stabilizer", "canon", "md_cano500"]` | 1.541 ms (5%) | | | | | `["stabilizer", "canon", "md_rref500"]` | 1.585 ms (5%) | | | | | `["stabilizer", "canon", "rref500"]` | 3.140 ms (5%) | | | | | `["stabilizer", "project", "destabilizer"]` | 18.475 μs (5%) | | 288 bytes (1%) | 5 | | `["stabilizer", "project", "stabilizer"]` | 8.887 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "tensor", "diag_pow5_20"]` | 1.257 ms (5%) | | 23.97 MiB (1%) | 34 | | `["stabilizer", "tensor", "pow5_20"]` | 2.769 μs (5%) | | 6.44 KiB (1%) | 31 | | `["stabilizer", "trace", "destabilizer"]` | 33.192 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "trace", "stabilizer"]` | 24.455 μs (5%) | | 112 bytes (1%) | 3 | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 1779 s 0 s 162 s 7262 s 0 s #2 0 MHz 2054 s 0 s 172 s 6979 s 0 s #3 0 MHz 2571 s 0 s 188 s 6450 s 0 s #4 0 MHz 2907 s 0 s 186 s 6108 s 0 s Memory: 15.606494903564453 GB (13357.609375 MB free) Uptime: 925.16 sec Load Avg: 1.14 1.07 0.76 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Baseline result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmark: 18 Apr 2024 - 15:22 * Package commit: 64f695 * Julia commit: 0f7674 * Julia command flags: None * Environment variables: None ## Results Below is a table of this job's results, obtained by running the benchmarks. The values listed in the `ID` column have the structure `[parent_group, child_group, ..., key]`, and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero. | ID | time | GC time | memory | allocations | |--------------------------------------------------------------------|----------------:|----------:|----------------:|------------:| | `["circuitsim", "mctrajectories", "q1001_r1"]` | 15.121 ms (5%) | | 500.77 KiB (1%) | 18019 | | `["circuitsim", "mctrajectories", "q101_r1"]` | 170.857 μs (5%) | | 50.53 KiB (1%) | 1818 | | `["circuitsim", "mctrajectories_sumtype", "q1001_r1"]` | 13.512 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_sumtype", "q101_r1"]` | 118.721 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "mctrajectories_union", "q1001_r1"]` | 13.205 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_union", "q101_r1"]` | 117.318 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "pftrajectories", "q1001_r1"]` | 63.258 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r100"]` | 173.763 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r10000"]` | 1.103 ms (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q101_r1"]` | 6.381 μs (5%) | | 9.42 KiB (1%) | 201 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r1"]` | 60.432 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r100"]` | 169.104 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]` | 1.104 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]` | 5.828 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q101_r1"]` | 6.061 μs (5%) | | 9.47 KiB (1%) | 202 | | `["circuitsim", "pftrajectories_union", "q1001_r1"]` | 23.213 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r100"]` | 132.997 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r10000"]` | 1.055 ms (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q101_r1"]` | 2.394 μs (5%) | | 96 bytes (1%) | 2 | | `["clifford", "dense", "cnot250_on_dense500_destab"]` | 11.491 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_dense500_stab"]` | 5.740 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_destab"]` | 1.121 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_stab"]` | 560.604 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot_on_dense500_destab"]` | 49.933 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_dense500_stab"]` | 24.456 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_destab"]` | 26.239 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_stab"]` | 13.735 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "dense500_on_dense500_destab"]` | 11.514 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_dense500_stab"]` | 5.744 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_destab"]` | 1.123 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_stab"]` | 560.611 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "symbolic", "cnot250_on_dense500_destab"]` | 1.432 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_dense500_stab"]` | 688.320 μs (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_destab"]` | 1.165 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_stab"]` | 570.632 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_destab"]` | 4.749 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_stab"]` | 2.304 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_destab"]` | 4.689 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_stab"]` | 2.294 μs (5%) | | | | | `["ecc", "evaluate_decoder", "shor_bp_comm"]` | 1.845 ms (5%) | | 1.78 MiB (1%) | 42707 | | `["ecc", "evaluate_decoder", "shor_bp_naivesyn"]` | 4.174 ms (5%) | | 3.61 MiB (1%) | 81648 | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 4.947 ms (5%) | | 3.77 MiB (1%) | 84162 | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 65.328 ms (5%) | | 4.76 MiB (1%) | 144207 | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 150.995 ms (5%) | | 9.57 MiB (1%) | 284018 | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 119.841 ms (5%) | | 9.66 MiB (1%) | 285104 | | `["ecc", "evaluate_decoder", "shor_table_comm"]` | 255.975 μs (5%) | | 241.42 KiB (1%) | 4593 | | `["ecc", "evaluate_decoder", "shor_table_naivesyn"]` | 863.045 μs (5%) | | 334.20 KiB (1%) | 4790 | | `["ecc", "evaluate_decoder", "shor_table_shorsyn"]` | 1.264 ms (5%) | | 427.73 KiB (1%) | 5876 | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 693.792 ms (5%) | 11.466 ms | 169.53 MiB (1%) | 1016786 | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 1.370 s (5%) | 24.615 ms | 343.32 MiB (1%) | 2080925 | | `["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]` | 1.418 s (5%) | 28.781 ms | 356.86 MiB (1%) | 2178180 | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 484.602 ms (5%) | | 8.11 MiB (1%) | 163696 | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 1.001 s (5%) | | 18.04 MiB (1%) | 360388 | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 953.327 ms (5%) | | 20.56 MiB (1%) | 392802 | | `["ecc", "evaluate_decoder", "toric8_pymatch_comm"]` | 4.271 ms (5%) | | 1.95 MiB (1%) | 23176 | | `["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]` | 12.474 ms (5%) | | 5.58 MiB (1%) | 79355 | | `["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]` | 21.033 ms (5%) | | 8.09 MiB (1%) | 111769 | | `["ecc", "evaluate_decoder", "toric8_table_comm"]` | 2.628 ms (5%) | | 1.64 MiB (1%) | 23082 | | `["ecc", "evaluate_decoder", "toric8_table_naivesyn"]` | 10.179 ms (5%) | | 4.93 MiB (1%) | 79160 | | `["ecc", "evaluate_decoder", "toric8_table_shorsyn"]` | 18.726 ms (5%) | | 7.44 MiB (1%) | 111574 | | `["pauli", "mul", "100"]` | 14.396 ns (5%) | | | | | `["pauli", "mul", "1000"]` | 18.209 ns (5%) | | | | | `["pauli", "mul", "100000"]` | 731.992 ns (5%) | | | | | `["pauli", "mul", "20000000"]` | 196.236 μs (5%) | | | | | `["stabilizer", "canon", "cano500"]` | 3.081 ms (5%) | | | | | `["stabilizer", "canon", "diag_cano500"]` | 676.397 μs (5%) | | | | | `["stabilizer", "canon", "diag_gott500"]` | 4.163 ms (5%) | | 5.19 MiB (1%) | 34521 | | `["stabilizer", "canon", "diag_rref500"]` | 665.978 μs (5%) | | | | | `["stabilizer", "canon", "gott500"]` | 6.575 ms (5%) | | 5.19 MiB (1%) | 34531 | | `["stabilizer", "canon", "md_cano500"]` | 1.543 ms (5%) | | | | | `["stabilizer", "canon", "md_rref500"]` | 1.556 ms (5%) | | | | | `["stabilizer", "canon", "rref500"]` | 3.142 ms (5%) | | | | | `["stabilizer", "project", "destabilizer"]` | 18.444 μs (5%) | | 288 bytes (1%) | 5 | | `["stabilizer", "project", "stabilizer"]` | 8.866 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "tensor", "diag_pow5_20"]` | 1.266 ms (5%) | | 23.97 MiB (1%) | 34 | | `["stabilizer", "tensor", "pow5_20"]` | 2.751 μs (5%) | | 6.44 KiB (1%) | 31 | | `["stabilizer", "trace", "destabilizer"]` | 31.439 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "trace", "stabilizer"]` | 24.205 μs (5%) | | 112 bytes (1%) | 3 | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 2528 s 0 s 261 s 10527 s 0 s #2 0 MHz 3405 s 0 s 265 s 9654 s 0 s #3 0 MHz 3625 s 0 s 267 s 9440 s 0 s #4 0 MHz 3969 s 0 s 264 s 9093 s 0 s Memory: 15.606494903564453 GB (13249.9453125 MB free) Uptime: 1338.58 sec Load Avg: 1.15 1.12 0.92 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Runtime information | Runtime Info | | |:--|:--| | BLAS #threads | 2 | | `BLAS.vendor()` | `lbt` | | `Sys.CPU_THREADS` | 4 | `lscpu` output: Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 48 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Vendor ID: AuthenticAMD Model name: AMD EPYC 7763 64-Core Processor CPU family: 25 Model: 1 Thread(s) per core: 2 Core(s) per socket: 2 Socket(s): 1 Stepping: 1 BogoMIPS: 4890.86 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm Virtualization: AMD-V Hypervisor vendor: Microsoft Virtualization type: full L1d cache: 64 KiB (2 instances) L1i cache: 64 KiB (2 instances) L2 cache: 1 MiB (2 instances) L3 cache: 32 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-3 Vulnerability Gather data sampling: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode Vulnerability Spec store bypass: Vulnerable Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected | Cpu Property | Value | |:------------------ |:---------------------------------------------------------- | | Brand | AMD EPYC 7763 64-Core Processor | | Vendor | :AMD | | Architecture | :Unknown | | Model | Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00 | | Cores | 16 physical cores, 16 logical cores (on executing CPU) | | | No Hyperthreading hardware capability detected | | Clock Frequencies | Not supported by CPU | | Data Cache | Level 1:3 : (32, 512, 32768) kbytes | | | 64 byte cache line size | | Address Size | 48 bits virtual, 48 bits physical | | SIMD | 256 bit = 32 byte max. SIMD vector size | | Time Stamp Counter | TSC is accessible via `rdtsc` | | | TSC runs at constant rate (invariant from clock frequency) | | Perf. Monitoring | Performance Monitoring Counters (PMC) are not supported | | Hypervisor | Yes, Microsoft |Benchmark Result
# Judge result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmarks: - Target: 18 Apr 2024 - 15:45 - Baseline: 18 Apr 2024 - 15:52 * Package commits: - Target: d01f41 - Baseline: 64f695 * Julia commits: - Target: 0f7674 - Baseline: 0f7674 * Julia command flags: - Target: None - Baseline: None * Environment variables: - Target: None - Baseline: None ## Results A ratio greater than `1.0` denotes a possible regression (marked with :x:), while a ratio less than `1.0` denotes a possible improvement (marked with :white_check_mark:). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds). | ID | time ratio | memory ratio | |--------------------------------------------------------------------|------------------------------|------------------------------| | `["clifford", "dense", "cnot_on_dense500_stab"]` | 1.06 (5%) :x: | 1.00 (1%) | | `["clifford", "dense", "dense500_on_diag500_destab"]` | 0.87 (5%) :white_check_mark: | 1.00 (1%) | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 0.92 (5%) :white_check_mark: | 0.98 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 0.34 (5%) :white_check_mark: | 0.83 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 0.33 (5%) :white_check_mark: | 0.83 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 0.38 (5%) :white_check_mark: | 0.83 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 1.01 (5%) | 1.02 (1%) :x: | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 0.97 (5%) | 0.97 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 0.80 (5%) :white_check_mark: | 0.90 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 0.83 (5%) :white_check_mark: | 0.91 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 0.83 (5%) :white_check_mark: | 0.92 (1%) :white_check_mark: | | `["pauli", "mul", "1000"]` | 1.07 (5%) :x: | 1.00 (1%) | | `["stabilizer", "tensor", "diag_pow5_20"]` | 0.91 (5%) :white_check_mark: | 1.00 (1%) | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ### Target ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 2081 s 0 s 169 s 6834 s 0 s #2 0 MHz 2125 s 0 s 185 s 6784 s 0 s #3 0 MHz 2143 s 0 s 191 s 6744 s 0 s #4 0 MHz 2729 s 0 s 192 s 6162 s 0 s Memory: 15.606494903564453 GB (13380.7578125 MB free) Uptime: 913.16 sec Load Avg: 1.03 1.01 0.71 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` ### Baseline ``` Julia Version 1.12.0-DEV.361 Commit 0f7674e2266 (2024-04-17 14:44 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 3162 s 0 s 269 s 9933 s 0 s #2 0 MHz 3507 s 0 s 282 s 9586 s 0 s #3 0 MHz 2978 s 0 s 281 s 10099 s 0 s #4 0 MHz 3861 s 0 s 295 s 9207 s 0 s Memory: 15.606494903564453 GB (13273.22265625 MB free) Uptime: 1342.65 sec Load Avg: 1.16 1.09 0.88 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Target result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmark: 18 Apr 2024 - 15:45 * Package commit: d01f41 * Julia commit: 0f7674 * Julia command flags: None * Environment variables: None ## Results Below is a table of this job's results, obtained by running the benchmarks. The values listed in the `ID` column have the structure `[parent_group, child_group, ..., key]`, and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero. | ID | time | GC time | memory | allocations | |--------------------------------------------------------------------|----------------:|----------:|----------------:|------------:| | `["circuitsim", "mctrajectories", "q1001_r1"]` | 15.191 ms (5%) | | 500.77 KiB (1%) | 18019 | | `["circuitsim", "mctrajectories", "q101_r1"]` | 172.271 μs (5%) | | 50.53 KiB (1%) | 1818 | | `["circuitsim", "mctrajectories_sumtype", "q1001_r1"]` | 13.316 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_sumtype", "q101_r1"]` | 119.092 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "mctrajectories_union", "q1001_r1"]` | 13.246 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_union", "q101_r1"]` | 117.910 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "pftrajectories", "q1001_r1"]` | 59.982 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r100"]` | 177.401 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r10000"]` | 1.146 ms (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q101_r1"]` | 6.031 μs (5%) | | 9.42 KiB (1%) | 201 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r1"]` | 44.583 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r100"]` | 158.977 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]` | 1.129 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]` | 5.872 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q101_r1"]` | 4.448 μs (5%) | | 9.47 KiB (1%) | 202 | | `["circuitsim", "pftrajectories_union", "q1001_r1"]` | 22.913 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r100"]` | 136.535 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r10000"]` | 1.098 ms (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q101_r1"]` | 2.364 μs (5%) | | 96 bytes (1%) | 2 | | `["clifford", "dense", "cnot250_on_dense500_destab"]` | 11.658 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_dense500_stab"]` | 5.849 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_destab"]` | 1.123 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_stab"]` | 560.655 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot_on_dense500_destab"]` | 51.215 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_dense500_stab"]` | 24.195 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_destab"]` | 26.329 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_stab"]` | 13.616 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "dense500_on_dense500_destab"]` | 11.653 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_dense500_stab"]` | 5.855 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_destab"]` | 980.047 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_stab"]` | 560.655 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "symbolic", "cnot250_on_dense500_destab"]` | 1.440 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_dense500_stab"]` | 686.199 μs (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_destab"]` | 1.160 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_stab"]` | 569.271 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_destab"]` | 4.739 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_stab"]` | 2.314 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_destab"]` | 4.658 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_stab"]` | 2.284 μs (5%) | | | | | `["ecc", "evaluate_decoder", "shor_bp_comm"]` | 1.820 ms (5%) | | 1.77 MiB (1%) | 42665 | | `["ecc", "evaluate_decoder", "shor_bp_naivesyn"]` | 4.169 ms (5%) | | 3.61 MiB (1%) | 81704 | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 4.548 ms (5%) | | 3.71 MiB (1%) | 82902 | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 24.183 ms (5%) | | 3.97 MiB (1%) | 116082 | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 48.685 ms (5%) | | 7.97 MiB (1%) | 227768 | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 49.012 ms (5%) | | 8.06 MiB (1%) | 228854 | | `["ecc", "evaluate_decoder", "shor_table_comm"]` | 255.006 μs (5%) | | 241.42 KiB (1%) | 4593 | | `["ecc", "evaluate_decoder", "shor_table_naivesyn"]` | 870.613 μs (5%) | | 334.20 KiB (1%) | 4790 | | `["ecc", "evaluate_decoder", "shor_table_shorsyn"]` | 1.264 ms (5%) | | 427.73 KiB (1%) | 5876 | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 676.475 ms (5%) | 13.812 ms | 170.14 MiB (1%) | 1020398 | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 1.377 s (5%) | 27.679 ms | 350.41 MiB (1%) | 2122596 | | `["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]` | 1.382 s (5%) | 29.151 ms | 351.42 MiB (1%) | 2146183 | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 397.337 ms (5%) | | 7.31 MiB (1%) | 135571 | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 809.669 ms (5%) | | 16.45 MiB (1%) | 304138 | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 825.844 ms (5%) | | 18.96 MiB (1%) | 336552 | | `["ecc", "evaluate_decoder", "toric8_pymatch_comm"]` | 4.207 ms (5%) | | 1.95 MiB (1%) | 23176 | | `["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]` | 12.550 ms (5%) | | 5.58 MiB (1%) | 79355 | | `["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]` | 20.928 ms (5%) | | 8.09 MiB (1%) | 111769 | | `["ecc", "evaluate_decoder", "toric8_table_comm"]` | 2.634 ms (5%) | | 1.64 MiB (1%) | 23082 | | `["ecc", "evaluate_decoder", "toric8_table_naivesyn"]` | 10.279 ms (5%) | | 4.93 MiB (1%) | 79160 | | `["ecc", "evaluate_decoder", "toric8_table_shorsyn"]` | 18.603 ms (5%) | | 7.44 MiB (1%) | 111574 | | `["pauli", "mul", "100"]` | 14.827 ns (5%) | | | | | `["pauli", "mul", "1000"]` | 18.831 ns (5%) | | | | | `["pauli", "mul", "100000"]` | 731.835 ns (5%) | | | | | `["pauli", "mul", "20000000"]` | 182.841 μs (5%) | | | | | `["stabilizer", "canon", "cano500"]` | 3.046 ms (5%) | | | | | `["stabilizer", "canon", "diag_cano500"]` | 708.661 μs (5%) | | | | | `["stabilizer", "canon", "diag_gott500"]` | 4.037 ms (5%) | | 5.19 MiB (1%) | 34521 | | `["stabilizer", "canon", "diag_rref500"]` | 704.153 μs (5%) | | | | | `["stabilizer", "canon", "gott500"]` | 6.382 ms (5%) | | 5.19 MiB (1%) | 34531 | | `["stabilizer", "canon", "md_cano500"]` | 1.544 ms (5%) | | | | | `["stabilizer", "canon", "md_rref500"]` | 1.574 ms (5%) | | | | | `["stabilizer", "canon", "rref500"]` | 3.095 ms (5%) | | | | | `["stabilizer", "project", "destabilizer"]` | 18.374 μs (5%) | | 288 bytes (1%) | 5 | | `["stabilizer", "project", "stabilizer"]` | 9.468 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "tensor", "diag_pow5_20"]` | 1.091 ms (5%) | | 23.97 MiB (1%) | 34 | | `["stabilizer", "tensor", "pow5_20"]` | 2.809 μs (5%) | | 6.44 KiB (1%) | 31 | | `["stabilizer", "trace", "destabilizer"]` | 31.349 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "trace", "stabilizer"]` | 24.255 μs (5%) | | 112 bytes (1%) | 3 | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - ...*[Comment body truncated]*