QuantumSavory / QuantumClifford.jl

Clifford circuits, graph states, and other quantum Stabilizer formalism tools.
MIT License
113 stars 45 forks source link

improvements to the non-clifford functionality #259

Open Krastanov opened 5 months ago

Krastanov commented 5 months ago

@hongyehu , this is what be built yesterday during the meeting, with a bit of polish. I tried to make the commits well separated and easy to read one by one. Hopefully this will make it easier to follow as you do not need to read the entire diff.

I will wait for tests to pass and then merge. I will add a long list of issues to track necessary enhancements.

github-actions[bot] commented 5 months ago
Benchmark Result # Judge result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmarks: - Target: 12 Apr 2024 - 13:38 - Baseline: 12 Apr 2024 - 13:45 * Package commits: - Target: 4fef66 - Baseline: 64f695 * Julia commits: - Target: 1ae41a - Baseline: 1ae41a * Julia command flags: - Target: None - Baseline: None * Environment variables: - Target: None - Baseline: None ## Results A ratio greater than `1.0` denotes a possible regression (marked with :x:), while a ratio less than `1.0` denotes a possible improvement (marked with :white_check_mark:). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds). | ID | time ratio | memory ratio | |--------------------------------------------------------------------|------------------------------|------------------------------| | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 0.93 (5%) :white_check_mark: | 0.98 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 1.10 (5%) :x: | 1.00 (1%) | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 0.75 (5%) :white_check_mark: | 1.00 (1%) | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 0.83 (5%) :white_check_mark: | 1.00 (1%) | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 1.03 (5%) | 1.04 (1%) :x: | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 1.03 (5%) | 1.03 (1%) :x: | | `["stabilizer", "canon", "diag_rref500"]` | 1.06 (5%) :x: | 1.00 (1%) | | `["stabilizer", "trace", "destabilizer"]` | 0.91 (5%) :white_check_mark: | 1.00 (1%) | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ### Target ``` Julia Version 1.12.0-DEV.334 Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 2544 s 0 s 289 s 6455 s 0 s #2 0 MHz 1822 s 0 s 288 s 7164 s 0 s #3 0 MHz 2181 s 0 s 292 s 6819 s 0 s #4 0 MHz 3122 s 0 s 272 s 5903 s 0 s Memory: 15.606498718261719 GB (13358.14453125 MB free) Uptime: 935.01 sec Load Avg: 1.0 1.09 0.8 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` ### Baseline ``` Julia Version 1.12.0-DEV.334 Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 3520 s 0 s 382 s 9741 s 0 s #2 0 MHz 2872 s 0 s 360 s 10400 s 0 s #3 0 MHz 3337 s 0 s 390 s 9915 s 0 s #4 0 MHz 4441 s 0 s 374 s 8831 s 0 s Memory: 15.606498718261719 GB (13207.7734375 MB free) Uptime: 1371.81 sec Load Avg: 1.05 1.1 0.94 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Target result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmark: 12 Apr 2024 - 13:38 * Package commit: 4fef66 * Julia commit: 1ae41a * Julia command flags: None * Environment variables: None ## Results Below is a table of this job's results, obtained by running the benchmarks. The values listed in the `ID` column have the structure `[parent_group, child_group, ..., key]`, and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero. | ID | time | GC time | memory | allocations | |--------------------------------------------------------------------|----------------:|----------:|----------------:|------------:| | `["circuitsim", "mctrajectories", "q1001_r1"]` | 14.956 ms (5%) | | 500.77 KiB (1%) | 18019 | | `["circuitsim", "mctrajectories", "q101_r1"]` | 169.877 μs (5%) | | 50.53 KiB (1%) | 1818 | | `["circuitsim", "mctrajectories_sumtype", "q1001_r1"]` | 13.279 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_sumtype", "q101_r1"]` | 117.250 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "mctrajectories_union", "q1001_r1"]` | 12.998 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_union", "q101_r1"]` | 116.057 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "pftrajectories", "q1001_r1"]` | 62.076 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r100"]` | 173.353 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r10000"]` | 1.102 ms (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q101_r1"]` | 6.201 μs (5%) | | 9.42 KiB (1%) | 201 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r1"]` | 44.543 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r100"]` | 157.774 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]` | 1.084 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]` | 5.856 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q101_r1"]` | 4.518 μs (5%) | | 9.47 KiB (1%) | 202 | | `["circuitsim", "pftrajectories_union", "q1001_r1"]` | 22.622 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r100"]` | 132.046 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r10000"]` | 1.056 ms (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q101_r1"]` | 2.334 μs (5%) | | 96 bytes (1%) | 2 | | `["clifford", "dense", "cnot250_on_dense500_destab"]` | 11.823 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_dense500_stab"]` | 5.788 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_destab"]` | 1.123 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_stab"]` | 560.627 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot_on_dense500_destab"]` | 50.795 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_dense500_stab"]` | 24.326 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_destab"]` | 26.519 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_stab"]` | 13.676 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "dense500_on_dense500_destab"]` | 11.866 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_dense500_stab"]` | 5.783 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_destab"]` | 979.950 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_stab"]` | 560.638 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "symbolic", "cnot250_on_dense500_destab"]` | 1.449 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_dense500_stab"]` | 688.406 μs (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_destab"]` | 1.159 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_stab"]` | 569.814 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_destab"]` | 4.729 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_stab"]` | 2.304 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_destab"]` | 4.659 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_stab"]` | 2.295 μs (5%) | | | | | `["ecc", "evaluate_decoder", "shor_bp_comm"]` | 1.826 ms (5%) | | 1.78 MiB (1%) | 42707 | | `["ecc", "evaluate_decoder", "shor_bp_naivesyn"]` | 4.093 ms (5%) | | 3.61 MiB (1%) | 81494 | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 4.535 ms (5%) | | 3.72 MiB (1%) | 83014 | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 58.789 ms (5%) | | 4.76 MiB (1%) | 144207 | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 119.038 ms (5%) | | 9.57 MiB (1%) | 284018 | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 124.628 ms (5%) | | 9.66 MiB (1%) | 285104 | | `["ecc", "evaluate_decoder", "shor_table_comm"]` | 257.010 μs (5%) | | 241.42 KiB (1%) | 4593 | | `["ecc", "evaluate_decoder", "shor_table_naivesyn"]` | 896.275 μs (5%) | | 334.20 KiB (1%) | 4790 | | `["ecc", "evaluate_decoder", "shor_table_shorsyn"]` | 1.296 ms (5%) | | 427.73 KiB (1%) | 5876 | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 706.062 ms (5%) | 18.157 ms | 174.51 MiB (1%) | 1046123 | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 1.429 s (5%) | 25.180 ms | 357.36 MiB (1%) | 2163476 | | `["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]` | 1.412 s (5%) | 27.075 ms | 350.81 MiB (1%) | 2142613 | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 459.145 ms (5%) | | 8.11 MiB (1%) | 163696 | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 968.377 ms (5%) | | 18.04 MiB (1%) | 360388 | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 983.377 ms (5%) | | 20.56 MiB (1%) | 392802 | | `["ecc", "evaluate_decoder", "toric8_pymatch_comm"]` | 4.156 ms (5%) | | 1.95 MiB (1%) | 23176 | | `["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]` | 12.443 ms (5%) | | 5.58 MiB (1%) | 79355 | | `["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]` | 21.042 ms (5%) | | 8.09 MiB (1%) | 111769 | | `["ecc", "evaluate_decoder", "toric8_table_comm"]` | 2.613 ms (5%) | | 1.64 MiB (1%) | 23082 | | `["ecc", "evaluate_decoder", "toric8_table_naivesyn"]` | 10.148 ms (5%) | | 4.93 MiB (1%) | 79160 | | `["ecc", "evaluate_decoder", "toric8_table_shorsyn"]` | 18.707 ms (5%) | | 7.44 MiB (1%) | 111574 | | `["pauli", "mul", "100"]` | 14.496 ns (5%) | | | | | `["pauli", "mul", "1000"]` | 17.628 ns (5%) | | | | | `["pauli", "mul", "100000"]` | 730.320 ns (5%) | | | | | `["pauli", "mul", "20000000"]` | 192.289 μs (5%) | | | | | `["stabilizer", "canon", "cano500"]` | 3.074 ms (5%) | | | | | `["stabilizer", "canon", "diag_cano500"]` | 709.174 μs (5%) | | | | | `["stabilizer", "canon", "diag_gott500"]` | 4.078 ms (5%) | | 5.19 MiB (1%) | 34521 | | `["stabilizer", "canon", "diag_rref500"]` | 675.983 μs (5%) | | | | | `["stabilizer", "canon", "gott500"]` | 6.480 ms (5%) | | 5.19 MiB (1%) | 34531 | | `["stabilizer", "canon", "md_cano500"]` | 1.538 ms (5%) | | | | | `["stabilizer", "canon", "md_rref500"]` | 1.568 ms (5%) | | | | | `["stabilizer", "canon", "rref500"]` | 3.148 ms (5%) | | | | | `["stabilizer", "project", "destabilizer"]` | 18.485 μs (5%) | | 288 bytes (1%) | 5 | | `["stabilizer", "project", "stabilizer"]` | 8.906 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "tensor", "diag_pow5_20"]` | 1.405 ms (5%) | | 23.97 MiB (1%) | 34 | | `["stabilizer", "tensor", "pow5_20"]` | 2.948 μs (5%) | | 6.44 KiB (1%) | 31 | | `["stabilizer", "trace", "destabilizer"]` | 30.918 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "trace", "stabilizer"]` | 23.854 μs (5%) | | 112 bytes (1%) | 3 | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ``` Julia Version 1.12.0-DEV.334 Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 2544 s 0 s 289 s 6455 s 0 s #2 0 MHz 1822 s 0 s 288 s 7164 s 0 s #3 0 MHz 2181 s 0 s 292 s 6819 s 0 s #4 0 MHz 3122 s 0 s 272 s 5903 s 0 s Memory: 15.606498718261719 GB (13358.14453125 MB free) Uptime: 935.01 sec Load Avg: 1.0 1.09 0.8 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Baseline result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmark: 12 Apr 2024 - 13:45 * Package commit: 64f695 * Julia commit: 1ae41a * Julia command flags: None * Environment variables: None ## Results Below is a table of this job's results, obtained by running the benchmarks. The values listed in the `ID` column have the structure `[parent_group, child_group, ..., key]`, and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero. | ID | time | GC time | memory | allocations | |--------------------------------------------------------------------|----------------:|----------:|----------------:|------------:| | `["circuitsim", "mctrajectories", "q1001_r1"]` | 15.339 ms (5%) | | 500.77 KiB (1%) | 18019 | | `["circuitsim", "mctrajectories", "q101_r1"]` | 171.411 μs (5%) | | 50.53 KiB (1%) | 1818 | | `["circuitsim", "mctrajectories_sumtype", "q1001_r1"]` | 13.385 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_sumtype", "q101_r1"]` | 118.872 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "mctrajectories_union", "q1001_r1"]` | 13.479 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_union", "q101_r1"]` | 117.109 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "pftrajectories", "q1001_r1"]` | 59.482 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r100"]` | 169.527 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r10000"]` | 1.094 ms (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q101_r1"]` | 6.001 μs (5%) | | 9.42 KiB (1%) | 201 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r1"]` | 44.292 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r100"]` | 155.120 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]` | 1.086 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]` | 5.862 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q101_r1"]` | 4.489 μs (5%) | | 9.47 KiB (1%) | 202 | | `["circuitsim", "pftrajectories_union", "q1001_r1"]` | 22.612 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r100"]` | 131.897 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r10000"]` | 1.051 ms (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q101_r1"]` | 2.334 μs (5%) | | 96 bytes (1%) | 2 | | `["clifford", "dense", "cnot250_on_dense500_destab"]` | 11.626 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_dense500_stab"]` | 5.710 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_destab"]` | 1.122 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_stab"]` | 560.507 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot_on_dense500_destab"]` | 50.956 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_dense500_stab"]` | 24.385 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_destab"]` | 26.259 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_stab"]` | 13.785 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "dense500_on_dense500_destab"]` | 11.632 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_dense500_stab"]` | 5.705 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_destab"]` | 980.362 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_stab"]` | 560.507 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "symbolic", "cnot250_on_dense500_destab"]` | 1.441 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_dense500_stab"]` | 685.972 μs (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_destab"]` | 1.161 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_stab"]` | 569.925 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_destab"]` | 4.759 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_stab"]` | 2.294 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_destab"]` | 4.668 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_stab"]` | 2.294 μs (5%) | | | | | `["ecc", "evaluate_decoder", "shor_bp_comm"]` | 1.810 ms (5%) | | 1.77 MiB (1%) | 42567 | | `["ecc", "evaluate_decoder", "shor_bp_naivesyn"]` | 4.089 ms (5%) | | 3.61 MiB (1%) | 81480 | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 4.890 ms (5%) | | 3.80 MiB (1%) | 84890 | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 53.401 ms (5%) | | 4.76 MiB (1%) | 144207 | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 159.039 ms (5%) | | 9.57 MiB (1%) | 284018 | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 150.257 ms (5%) | | 9.66 MiB (1%) | 285104 | | `["ecc", "evaluate_decoder", "shor_table_comm"]` | 255.052 μs (5%) | | 241.42 KiB (1%) | 4593 | | `["ecc", "evaluate_decoder", "shor_table_naivesyn"]` | 865.618 μs (5%) | | 334.20 KiB (1%) | 4790 | | `["ecc", "evaluate_decoder", "shor_table_shorsyn"]` | 1.264 ms (5%) | | 427.73 KiB (1%) | 5876 | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 684.192 ms (5%) | 13.009 ms | 168.33 MiB (1%) | 1009765 | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 1.387 s (5%) | 23.748 ms | 347.70 MiB (1%) | 2106685 | | `["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]` | 1.406 s (5%) | 32.426 ms | 352.19 MiB (1%) | 2150726 | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 479.316 ms (5%) | | 8.11 MiB (1%) | 163696 | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 1.000 s (5%) | | 18.04 MiB (1%) | 360388 | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 996.243 ms (5%) | | 20.56 MiB (1%) | 392802 | | `["ecc", "evaluate_decoder", "toric8_pymatch_comm"]` | 4.263 ms (5%) | | 1.95 MiB (1%) | 23176 | | `["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]` | 12.491 ms (5%) | | 5.58 MiB (1%) | 79355 | | `["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]` | 20.958 ms (5%) | | 8.09 MiB (1%) | 111769 | | `["ecc", "evaluate_decoder", "toric8_table_comm"]` | 2.672 ms (5%) | | 1.64 MiB (1%) | 23082 | | `["ecc", "evaluate_decoder", "toric8_table_naivesyn"]` | 10.177 ms (5%) | | 4.93 MiB (1%) | 79160 | | `["ecc", "evaluate_decoder", "toric8_table_shorsyn"]` | 18.742 ms (5%) | | 7.44 MiB (1%) | 111574 | | `["pauli", "mul", "100"]` | 14.597 ns (5%) | | | | | `["pauli", "mul", "1000"]` | 17.899 ns (5%) | | | | | `["pauli", "mul", "100000"]` | 731.848 ns (5%) | | | | | `["pauli", "mul", "20000000"]` | 202.258 μs (5%) | | | | | `["stabilizer", "canon", "cano500"]` | 3.083 ms (5%) | | | | | `["stabilizer", "canon", "diag_cano500"]` | 676.415 μs (5%) | | | | | `["stabilizer", "canon", "diag_gott500"]` | 4.155 ms (5%) | | 5.19 MiB (1%) | 34521 | | `["stabilizer", "canon", "diag_rref500"]` | 639.054 μs (5%) | | | | | `["stabilizer", "canon", "gott500"]` | 6.540 ms (5%) | | 5.19 MiB (1%) | 34531 | | `["stabilizer", "canon", "md_cano500"]` | 1.583 ms (5%) | | | | | `["stabilizer", "canon", "md_rref500"]` | 1.625 ms (5%) | | | | | `["stabilizer", "canon", "rref500"]` | 3.137 ms (5%) | | | | | `["stabilizer", "project", "destabilizer"]` | 18.534 μs (5%) | | 288 bytes (1%) | 5 | | `["stabilizer", "project", "stabilizer"]` | 9.027 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "tensor", "diag_pow5_20"]` | 1.436 ms (5%) | | 23.97 MiB (1%) | 34 | | `["stabilizer", "tensor", "pow5_20"]` | 2.829 μs (5%) | | 6.44 KiB (1%) | 31 | | `["stabilizer", "trace", "destabilizer"]` | 34.014 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "trace", "stabilizer"]` | 24.235 μs (5%) | | 112 bytes (1%) | 3 | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ``` Julia Version 1.12.0-DEV.334 Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 3520 s 0 s 382 s 9741 s 0 s #2 0 MHz 2872 s 0 s 360 s 10400 s 0 s #3 0 MHz 3337 s 0 s 390 s 9915 s 0 s #4 0 MHz 4441 s 0 s 374 s 8831 s 0 s Memory: 15.606498718261719 GB (13207.7734375 MB free) Uptime: 1371.81 sec Load Avg: 1.05 1.1 0.94 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Runtime information | Runtime Info | | |:--|:--| | BLAS #threads | 2 | | `BLAS.vendor()` | `lbt` | | `Sys.CPU_THREADS` | 4 | `lscpu` output: Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 48 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Vendor ID: AuthenticAMD Model name: AMD EPYC 7763 64-Core Processor CPU family: 25 Model: 1 Thread(s) per core: 2 Core(s) per socket: 2 Socket(s): 1 Stepping: 1 BogoMIPS: 4890.85 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm Virtualization: AMD-V Hypervisor vendor: Microsoft Virtualization type: full L1d cache: 64 KiB (2 instances) L1i cache: 64 KiB (2 instances) L2 cache: 1 MiB (2 instances) L3 cache: 32 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-3 Vulnerability Gather data sampling: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode Vulnerability Spec store bypass: Vulnerable Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected | Cpu Property | Value | |:------------------ |:---------------------------------------------------------- | | Brand | AMD EPYC 7763 64-Core Processor | | Vendor | :AMD | | Architecture | :Unknown | | Model | Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00 | | Cores | 16 physical cores, 16 logical cores (on executing CPU) | | | No Hyperthreading hardware capability detected | | Clock Frequencies | Not supported by CPU | | Data Cache | Level 1:3 : (32, 512, 32768) kbytes | | | 64 byte cache line size | | Address Size | 48 bits virtual, 48 bits physical | | SIMD | 256 bit = 32 byte max. SIMD vector size | | Time Stamp Counter | TSC is accessible via `rdtsc` | | | TSC runs at constant rate (invariant from clock frequency) | | Perf. Monitoring | Performance Monitoring Counters (PMC) are not supported | | Hypervisor | Yes, Microsoft |
Benchmark Result # Judge result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmarks: - Target: 12 Apr 2024 - 13:48 - Baseline: 12 Apr 2024 - 13:55 * Package commits: - Target: cb53f3 - Baseline: 64f695 * Julia commits: - Target: 1ae41a - Baseline: 1ae41a * Julia command flags: - Target: None - Baseline: None * Environment variables: - Target: None - Baseline: None ## Results A ratio greater than `1.0` denotes a possible regression (marked with :x:), while a ratio less than `1.0` denotes a possible improvement (marked with :white_check_mark:). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds). | ID | time ratio | memory ratio | |--------------------------------------------------------------------|------------------------------|------------------------------| | `["circuitsim", "mctrajectories_sumtype", "q1001_r1"]` | 0.93 (5%) :white_check_mark: | 1.00 (1%) | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 0.90 (5%) :white_check_mark: | 0.98 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 1.08 (5%) :x: | 1.00 (1%) | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 0.88 (5%) :white_check_mark: | 1.00 (1%) | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 0.99 (5%) | 1.01 (1%) :x: | | `["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]` | 0.97 (5%) | 0.98 (1%) :white_check_mark: | | `["stabilizer", "canon", "diag_cano500"]` | 1.11 (5%) :x: | 1.00 (1%) | | `["stabilizer", "canon", "diag_rref500"]` | 0.87 (5%) :white_check_mark: | 1.00 (1%) | | `["stabilizer", "tensor", "diag_pow5_20"]` | 1.16 (5%) :x: | 1.00 (1%) | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ### Target ``` Julia Version 1.12.0-DEV.334 Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 2166 s 0 s 308 s 6606 s 0 s #2 0 MHz 2714 s 0 s 285 s 6107 s 0 s #3 0 MHz 2285 s 0 s 284 s 6498 s 0 s #4 0 MHz 2540 s 0 s 296 s 6242 s 0 s Memory: 15.606494903564453 GB (13380.4765625 MB free) Uptime: 915.57 sec Load Avg: 1.1 1.14 0.82 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` ### Baseline ``` Julia Version 1.12.0-DEV.334 Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 2989 s 0 s 415 s 9989 s 0 s #2 0 MHz 3811 s 0 s 387 s 9224 s 0 s #3 0 MHz 3449 s 0 s 368 s 9567 s 0 s #4 0 MHz 3914 s 0 s 393 s 9088 s 0 s Memory: 15.606494903564453 GB (13298.3828125 MB free) Uptime: 1348.76 sec Load Avg: 1.0 1.06 0.93 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Target result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmark: 12 Apr 2024 - 13:48 * Package commit: cb53f3 * Julia commit: 1ae41a * Julia command flags: None * Environment variables: None ## Results Below is a table of this job's results, obtained by running the benchmarks. The values listed in the `ID` column have the structure `[parent_group, child_group, ..., key]`, and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero. | ID | time | GC time | memory | allocations | |--------------------------------------------------------------------|----------------:|----------:|----------------:|------------:| | `["circuitsim", "mctrajectories", "q1001_r1"]` | 14.997 ms (5%) | | 500.77 KiB (1%) | 18019 | | `["circuitsim", "mctrajectories", "q101_r1"]` | 170.285 μs (5%) | | 50.53 KiB (1%) | 1818 | | `["circuitsim", "mctrajectories_sumtype", "q1001_r1"]` | 13.349 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_sumtype", "q101_r1"]` | 117.488 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "mctrajectories_union", "q1001_r1"]` | 13.203 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_union", "q101_r1"]` | 116.325 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "pftrajectories", "q1001_r1"]` | 59.601 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r100"]` | 169.855 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r10000"]` | 1.100 ms (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q101_r1"]` | 6.001 μs (5%) | | 9.42 KiB (1%) | 201 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r1"]` | 44.612 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r100"]` | 154.937 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]` | 1.085 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]` | 5.871 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q101_r1"]` | 4.538 μs (5%) | | 9.47 KiB (1%) | 202 | | `["circuitsim", "pftrajectories_union", "q1001_r1"]` | 22.612 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r100"]` | 132.617 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r10000"]` | 1.054 ms (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q101_r1"]` | 2.334 μs (5%) | | 96 bytes (1%) | 2 | | `["clifford", "dense", "cnot250_on_dense500_destab"]` | 11.815 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_dense500_stab"]` | 5.789 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_destab"]` | 1.122 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_stab"]` | 560.616 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot_on_dense500_destab"]` | 52.897 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_dense500_stab"]` | 23.132 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_destab"]` | 26.309 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_stab"]` | 13.294 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "dense500_on_dense500_destab"]` | 11.816 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_dense500_stab"]` | 5.794 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_destab"]` | 980.032 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_stab"]` | 560.626 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "symbolic", "cnot250_on_dense500_destab"]` | 1.434 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_dense500_stab"]` | 687.821 μs (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_destab"]` | 1.162 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_stab"]` | 569.535 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_destab"]` | 4.749 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_stab"]` | 2.294 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_destab"]` | 4.669 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_stab"]` | 2.294 μs (5%) | | | | | `["ecc", "evaluate_decoder", "shor_bp_comm"]` | 1.838 ms (5%) | | 1.76 MiB (1%) | 42455 | | `["ecc", "evaluate_decoder", "shor_bp_naivesyn"]` | 4.042 ms (5%) | | 3.62 MiB (1%) | 81718 | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 4.486 ms (5%) | | 3.71 MiB (1%) | 82944 | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 60.832 ms (5%) | | 4.76 MiB (1%) | 144207 | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 146.900 ms (5%) | | 9.57 MiB (1%) | 284018 | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 128.963 ms (5%) | | 9.66 MiB (1%) | 285104 | | `["ecc", "evaluate_decoder", "shor_table_comm"]` | 257.289 μs (5%) | | 241.42 KiB (1%) | 4593 | | `["ecc", "evaluate_decoder", "shor_table_naivesyn"]` | 869.387 μs (5%) | | 334.20 KiB (1%) | 4790 | | `["ecc", "evaluate_decoder", "shor_table_shorsyn"]` | 1.265 ms (5%) | | 427.73 KiB (1%) | 5876 | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 676.413 ms (5%) | 10.161 ms | 173.16 MiB (1%) | 1038164 | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 1.421 s (5%) | 24.163 ms | 352.70 MiB (1%) | 2136078 | | `["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]` | 1.377 s (5%) | 28.011 ms | 347.48 MiB (1%) | 2123041 | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 474.298 ms (5%) | | 8.11 MiB (1%) | 163696 | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 1.001 s (5%) | | 18.04 MiB (1%) | 360388 | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 1.010 s (5%) | | 20.56 MiB (1%) | 392802 | | `["ecc", "evaluate_decoder", "toric8_pymatch_comm"]` | 4.160 ms (5%) | | 1.95 MiB (1%) | 23176 | | `["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]` | 12.361 ms (5%) | | 5.58 MiB (1%) | 79355 | | `["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]` | 20.775 ms (5%) | | 8.09 MiB (1%) | 111769 | | `["ecc", "evaluate_decoder", "toric8_table_comm"]` | 2.604 ms (5%) | | 1.64 MiB (1%) | 23082 | | `["ecc", "evaluate_decoder", "toric8_table_naivesyn"]` | 10.088 ms (5%) | | 4.93 MiB (1%) | 79160 | | `["ecc", "evaluate_decoder", "toric8_table_shorsyn"]` | 18.614 ms (5%) | | 7.44 MiB (1%) | 111574 | | `["pauli", "mul", "100"]` | 14.204 ns (5%) | | | | | `["pauli", "mul", "1000"]` | 17.909 ns (5%) | | | | | `["pauli", "mul", "100000"]` | 712.881 ns (5%) | | | | | `["pauli", "mul", "20000000"]` | 196.774 μs (5%) | | | | | `["stabilizer", "canon", "cano500"]` | 3.095 ms (5%) | | | | | `["stabilizer", "canon", "diag_cano500"]` | 752.408 μs (5%) | | | | | `["stabilizer", "canon", "diag_gott500"]` | 4.069 ms (5%) | | 5.19 MiB (1%) | 34521 | | `["stabilizer", "canon", "diag_rref500"]` | 626.931 μs (5%) | | | | | `["stabilizer", "canon", "gott500"]` | 6.415 ms (5%) | | 5.19 MiB (1%) | 34531 | | `["stabilizer", "canon", "md_cano500"]` | 1.548 ms (5%) | | | | | `["stabilizer", "canon", "md_rref500"]` | 1.575 ms (5%) | | | | | `["stabilizer", "canon", "rref500"]` | 3.100 ms (5%) | | | | | `["stabilizer", "project", "destabilizer"]` | 18.525 μs (5%) | | 288 bytes (1%) | 5 | | `["stabilizer", "project", "stabilizer"]` | 9.377 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "tensor", "diag_pow5_20"]` | 1.355 ms (5%) | | 23.97 MiB (1%) | 34 | | `["stabilizer", "tensor", "pow5_20"]` | 2.899 μs (5%) | | 6.44 KiB (1%) | 31 | | `["stabilizer", "trace", "destabilizer"]` | 31.548 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "trace", "stabilizer"]` | 24.094 μs (5%) | | 112 bytes (1%) | 3 | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ``` Julia Version 1.12.0-DEV.334 Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 2166 s 0 s 308 s 6606 s 0 s #2 0 MHz 2714 s 0 s 285 s 6107 s 0 s #3 0 MHz 2285 s 0 s 284 s 6498 s 0 s #4 0 MHz 2540 s 0 s 296 s 6242 s 0 s Memory: 15.606494903564453 GB (13380.4765625 MB free) Uptime: 915.57 sec Load Avg: 1.1 1.14 0.82 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Baseline result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmark: 12 Apr 2024 - 13:55 * Package commit: 64f695 * Julia commit: 1ae41a * Julia command flags: None * Environment variables: None ## Results Below is a table of this job's results, obtained by running the benchmarks. The values listed in the `ID` column have the structure `[parent_group, child_group, ..., key]`, and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero. | ID | time | GC time | memory | allocations | |--------------------------------------------------------------------|----------------:|----------:|----------------:|------------:| | `["circuitsim", "mctrajectories", "q1001_r1"]` | 15.324 ms (5%) | | 500.77 KiB (1%) | 18019 | | `["circuitsim", "mctrajectories", "q101_r1"]` | 171.759 μs (5%) | | 50.53 KiB (1%) | 1818 | | `["circuitsim", "mctrajectories_sumtype", "q1001_r1"]` | 14.359 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_sumtype", "q101_r1"]` | 119.102 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "mctrajectories_union", "q1001_r1"]` | 13.265 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_union", "q101_r1"]` | 118.039 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "pftrajectories", "q1001_r1"]` | 59.681 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r100"]` | 170.397 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r10000"]` | 1.096 ms (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q101_r1"]` | 5.991 μs (5%) | | 9.42 KiB (1%) | 201 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r1"]` | 44.813 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r100"]` | 156.360 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]` | 1.091 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]` | 5.822 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q101_r1"]` | 4.458 μs (5%) | | 9.47 KiB (1%) | 202 | | `["circuitsim", "pftrajectories_union", "q1001_r1"]` | 23.223 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r100"]` | 133.188 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r10000"]` | 1.056 ms (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q101_r1"]` | 2.394 μs (5%) | | 96 bytes (1%) | 2 | | `["clifford", "dense", "cnot250_on_dense500_destab"]` | 11.747 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_dense500_stab"]` | 5.819 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_destab"]` | 1.122 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_stab"]` | 561.704 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot_on_dense500_destab"]` | 50.523 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_dense500_stab"]` | 23.363 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_destab"]` | 26.309 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_stab"]` | 13.314 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "dense500_on_dense500_destab"]` | 11.806 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_dense500_stab"]` | 5.843 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_destab"]` | 980.380 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_stab"]` | 561.704 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "symbolic", "cnot250_on_dense500_destab"]` | 1.434 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_dense500_stab"]` | 687.427 μs (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_destab"]` | 1.161 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_stab"]` | 569.809 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_destab"]` | 4.769 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_stab"]` | 2.294 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_destab"]` | 4.688 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_stab"]` | 2.294 μs (5%) | | | | | `["ecc", "evaluate_decoder", "shor_bp_comm"]` | 1.826 ms (5%) | | 1.77 MiB (1%) | 42721 | | `["ecc", "evaluate_decoder", "shor_bp_naivesyn"]` | 4.086 ms (5%) | | 3.61 MiB (1%) | 81690 | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 5.003 ms (5%) | | 3.79 MiB (1%) | 84792 | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 59.083 ms (5%) | | 4.76 MiB (1%) | 144207 | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 136.484 ms (5%) | | 9.57 MiB (1%) | 284018 | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 145.730 ms (5%) | | 9.66 MiB (1%) | 285104 | | `["ecc", "evaluate_decoder", "shor_table_comm"]` | 255.874 μs (5%) | | 241.42 KiB (1%) | 4593 | | `["ecc", "evaluate_decoder", "shor_table_naivesyn"]` | 887.918 μs (5%) | | 334.20 KiB (1%) | 4790 | | `["ecc", "evaluate_decoder", "shor_table_shorsyn"]` | 1.275 ms (5%) | | 427.73 KiB (1%) | 5876 | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 684.094 ms (5%) | 15.175 ms | 170.60 MiB (1%) | 1023114 | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 1.390 s (5%) | 19.896 ms | 350.72 MiB (1%) | 2124437 | | `["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]` | 1.427 s (5%) | 24.819 ms | 356.12 MiB (1%) | 2173840 | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 488.487 ms (5%) | | 8.11 MiB (1%) | 163696 | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 1.006 s (5%) | | 18.04 MiB (1%) | 360388 | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 1.002 s (5%) | | 20.56 MiB (1%) | 392802 | | `["ecc", "evaluate_decoder", "toric8_pymatch_comm"]` | 4.279 ms (5%) | | 1.95 MiB (1%) | 23176 | | `["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]` | 12.430 ms (5%) | | 5.58 MiB (1%) | 79355 | | `["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]` | 20.840 ms (5%) | | 8.09 MiB (1%) | 111769 | | `["ecc", "evaluate_decoder", "toric8_table_comm"]` | 2.643 ms (5%) | | 1.64 MiB (1%) | 23082 | | `["ecc", "evaluate_decoder", "toric8_table_naivesyn"]` | 10.061 ms (5%) | | 4.93 MiB (1%) | 79160 | | `["ecc", "evaluate_decoder", "toric8_table_shorsyn"]` | 18.486 ms (5%) | | 7.44 MiB (1%) | 111574 | | `["pauli", "mul", "100"]` | 14.234 ns (5%) | | | | | `["pauli", "mul", "1000"]` | 17.748 ns (5%) | | | | | `["pauli", "mul", "100000"]` | 721.866 ns (5%) | | | | | `["pauli", "mul", "20000000"]` | 203.167 μs (5%) | | | | | `["stabilizer", "canon", "cano500"]` | 3.106 ms (5%) | | | | | `["stabilizer", "canon", "diag_cano500"]` | 676.426 μs (5%) | | | | | `["stabilizer", "canon", "diag_gott500"]` | 4.039 ms (5%) | | 5.19 MiB (1%) | 34521 | | `["stabilizer", "canon", "diag_rref500"]` | 721.900 μs (5%) | | | | | `["stabilizer", "canon", "gott500"]` | 6.428 ms (5%) | | 5.19 MiB (1%) | 34531 | | `["stabilizer", "canon", "md_cano500"]` | 1.550 ms (5%) | | | | | `["stabilizer", "canon", "md_rref500"]` | 1.564 ms (5%) | | | | | `["stabilizer", "canon", "rref500"]` | 3.144 ms (5%) | | | | | `["stabilizer", "project", "destabilizer"]` | 18.655 μs (5%) | | 288 bytes (1%) | 5 | | `["stabilizer", "project", "stabilizer"]` | 9.257 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "tensor", "diag_pow5_20"]` | 1.171 ms (5%) | | 23.97 MiB (1%) | 34 | | `["stabilizer", "tensor", "pow5_20"]` | 2.817 μs (5%) | | 6.44 KiB (1%) | 31 | | `["stabilizer", "trace", "destabilizer"]` | 31.288 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "trace", "stabilizer"]` | 24.216 μs (5%) | | 112 bytes (1%) | 3 | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ``` Julia Version 1.12.0-DEV.334 Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 2989 s 0 s 415 s 9989 s 0 s #2 0 MHz 3811 s 0 s 387 s 9224 s 0 s #3 0 MHz 3449 s 0 s 368 s 9567 s 0 s #4 0 MHz 3914 s 0 s 393 s 9088 s 0 s Memory: 15.606494903564453 GB (13298.3828125 MB free) Uptime: 1348.76 sec Load Avg: 1.0 1.06 0.93 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Runtime information | Runtime Info | | |:--|:--| | BLAS #threads | 2 | | `BLAS.vendor()` | `lbt` | | `Sys.CPU_THREADS` | 4 | `lscpu` output: Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 48 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Vendor ID: AuthenticAMD Model name: AMD EPYC 7763 64-Core Processor CPU family: 25 Model: 1 Thread(s) per core: 2 Core(s) per socket: 2 Socket(s): 1 Stepping: 1 BogoMIPS: 4890.86 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm Virtualization: AMD-V Hypervisor vendor: Microsoft Virtualization type: full L1d cache: 64 KiB (2 instances) L1i cache: 64 KiB (2 instances) L2 cache: 1 MiB (2 instances) L3 cache: 32 MiB (1 instance) NUMA node(s): 1 NUMA node0 CPU(s): 0-3 Vulnerability Gather data sampling: Not affected Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode Vulnerability Spec store bypass: Vulnerable Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected | Cpu Property | Value | |:------------------ |:---------------------------------------------------------- | | Brand | AMD EPYC 7763 64-Core Processor | | Vendor | :AMD | | Architecture | :Unknown | | Model | Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00 | | Cores | 16 physical cores, 16 logical cores (on executing CPU) | | | No Hyperthreading hardware capability detected | | Clock Frequencies | Not supported by CPU | | Data Cache | Level 1:3 : (32, 512, 32768) kbytes | | | 64 byte cache line size | | Address Size | 48 bits virtual, 48 bits physical | | SIMD | 256 bit = 32 byte max. SIMD vector size | | Time Stamp Counter | TSC is accessible via `rdtsc` | | | TSC runs at constant rate (invariant from clock frequency) | | Perf. Monitoring | Performance Monitoring Counters (PMC) are not supported | | Hypervisor | Yes, Microsoft |
Benchmark Result # Judge result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmarks: - Target: 12 Apr 2024 - 13:56 - Baseline: 12 Apr 2024 - 14:04 * Package commits: - Target: 3d58e9 - Baseline: 64f695 * Julia commits: - Target: 1ae41a - Baseline: 1ae41a * Julia command flags: - Target: None - Baseline: None * Environment variables: - Target: None - Baseline: None ## Results A ratio greater than `1.0` denotes a possible regression (marked with :x:), while a ratio less than `1.0` denotes a possible improvement (marked with :white_check_mark:). Only significant results - results that indicate possible regressions or improvements - are shown below (thus, an empty table means that all benchmark results remained invariant between builds). | ID | time ratio | memory ratio | |--------------------------------------------------------------------|------------------------------|------------------------------| | `["circuitsim", "mctrajectories_sumtype", "q1001_r1"]` | 0.90 (5%) :white_check_mark: | 1.00 (1%) | | `["circuitsim", "mctrajectories_union", "q1001_r1"]` | 0.92 (5%) :white_check_mark: | 1.00 (1%) | | `["circuitsim", "pftrajectories_union", "q101_r1"]` | 0.92 (5%) :white_check_mark: | 1.00 (1%) | | `["clifford", "dense", "cnot250_on_dense500_destab"]` | 1.16 (5%) :x: | 1.00 (1%) | | `["clifford", "dense", "cnot250_on_diag500_stab"]` | 0.88 (5%) :white_check_mark: | 1.00 (1%) | | `["clifford", "dense", "dense500_on_diag500_stab"]` | 0.87 (5%) :white_check_mark: | 1.00 (1%) | | `["clifford", "symbolic", "cnot_on_dense500_stab"]` | 0.93 (5%) :white_check_mark: | 1.00 (1%) | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 0.90 (5%) :white_check_mark: | 0.98 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 1.12 (5%) :x: | 1.00 (1%) | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 0.86 (5%) :white_check_mark: | 1.00 (1%) | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 0.94 (5%) :white_check_mark: | 0.96 (1%) :white_check_mark: | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 1.00 (5%) | 1.01 (1%) :x: | | `["pauli", "mul", "100000"]` | 0.92 (5%) :white_check_mark: | 1.00 (1%) | | `["stabilizer", "canon", "diag_cano500"]` | 1.06 (5%) :x: | 1.00 (1%) | | `["stabilizer", "canon", "diag_rref500"]` | 1.15 (5%) :x: | 1.00 (1%) | | `["stabilizer", "canon", "md_cano500"]` | 1.06 (5%) :x: | 1.00 (1%) | | `["stabilizer", "project", "destabilizer"]` | 0.93 (5%) :white_check_mark: | 1.00 (1%) | | `["stabilizer", "project", "stabilizer"]` | 1.08 (5%) :x: | 1.00 (1%) | | `["stabilizer", "trace", "destabilizer"]` | 1.08 (5%) :x: | 1.00 (1%) | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ### Target ``` Julia Version 1.12.0-DEV.334 Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 2543 s 0 s 272 s 5972 s 0 s #2 0 MHz 2933 s 0 s 284 s 5595 s 0 s #3 0 MHz 2016 s 0 s 298 s 6477 s 0 s #4 0 MHz 1869 s 0 s 293 s 6632 s 0 s Memory: 15.606498718261719 GB (13355.90625 MB free) Uptime: 885.05 sec Load Avg: 1.08 1.09 0.76 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` ### Baseline ``` Julia Version 1.12.0-DEV.334 Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 3595 s 0 s 356 s 9082 s 0 s #2 0 MHz 3962 s 0 s 394 s 8704 s 0 s #3 0 MHz 3468 s 0 s 391 s 9179 s 0 s #4 0 MHz 2731 s 0 s 387 s 9923 s 0 s Memory: 15.606498718261719 GB (13284.78515625 MB free) Uptime: 1311.18 sec Load Avg: 1.09 1.13 0.93 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) ``` --- # Target result # Benchmark Report for */home/runner/work/QuantumClifford.jl/QuantumClifford.jl* ## Job Properties * Time of benchmark: 12 Apr 2024 - 13:56 * Package commit: 3d58e9 * Julia commit: 1ae41a * Julia command flags: None * Environment variables: None ## Results Below is a table of this job's results, obtained by running the benchmarks. The values listed in the `ID` column have the structure `[parent_group, child_group, ..., key]`, and can be used to index into the BaseBenchmarks suite to retrieve the corresponding benchmarks. The percentages accompanying time and memory values in the below table are noise tolerances. The "true" time/memory value for a given benchmark is expected to fall within this percentage of the reported value. An empty cell means that the value was zero. | ID | time | GC time | memory | allocations | |--------------------------------------------------------------------|----------------:|----------:|----------------:|------------:| | `["circuitsim", "mctrajectories", "q1001_r1"]` | 14.300 ms (5%) | | 500.77 KiB (1%) | 18019 | | `["circuitsim", "mctrajectories", "q101_r1"]` | 159.397 μs (5%) | | 50.53 KiB (1%) | 1818 | | `["circuitsim", "mctrajectories_sumtype", "q1001_r1"]` | 12.642 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_sumtype", "q101_r1"]` | 109.774 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "mctrajectories_union", "q1001_r1"]` | 12.614 ms (5%) | | 496 bytes (1%) | 9 | | `["circuitsim", "mctrajectories_union", "q101_r1"]` | 109.143 μs (5%) | | 256 bytes (1%) | 8 | | `["circuitsim", "pftrajectories", "q1001_r1"]` | 58.399 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r100"]` | 161.982 μs (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q1001_r10000"]` | 1.017 ms (5%) | | 93.80 KiB (1%) | 2001 | | `["circuitsim", "pftrajectories", "q101_r1"]` | 5.861 μs (5%) | | 9.42 KiB (1%) | 201 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r1"]` | 40.645 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r100"]` | 145.422 μs (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000"]` | 1.003 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"]` | 5.391 ms (5%) | | 93.84 KiB (1%) | 2002 | | `["circuitsim", "pftrajectories_sumtype", "q101_r1"]` | 4.117 μs (5%) | | 9.47 KiB (1%) | 202 | | `["circuitsim", "pftrajectories_union", "q1001_r1"]` | 20.858 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r100"]` | 123.748 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q1001_r10000"]` | 973.621 μs (5%) | | 96 bytes (1%) | 2 | | `["circuitsim", "pftrajectories_union", "q101_r1"]` | 2.164 μs (5%) | | 96 bytes (1%) | 2 | | `["clifford", "dense", "cnot250_on_dense500_destab"]` | 12.768 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_dense500_stab"]` | 5.441 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_destab"]` | 1.038 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot250_on_diag500_stab"]` | 455.187 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "cnot_on_dense500_destab"]` | 46.927 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_dense500_stab"]` | 21.681 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_destab"]` | 24.225 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "cnot_on_diag500_stab"]` | 12.133 μs (5%) | | 368 bytes (1%) | 10 | | `["clifford", "dense", "dense500_on_dense500_destab"]` | 10.692 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_dense500_stab"]` | 5.310 ms (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_destab"]` | 903.782 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "dense", "dense500_on_diag500_stab"]` | 451.490 μs (5%) | | 512 bytes (1%) | 8 | | `["clifford", "symbolic", "cnot250_on_dense500_destab"]` | 1.323 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_dense500_stab"]` | 634.651 μs (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_destab"]` | 1.076 ms (5%) | | | | | `["clifford", "symbolic", "cnot250_on_diag500_stab"]` | 525.618 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_destab"]` | 4.388 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_dense500_stab"]` | 2.123 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_destab"]` | 4.328 μs (5%) | | | | | `["clifford", "symbolic", "cnot_on_diag500_stab"]` | 2.113 μs (5%) | | | | | `["ecc", "evaluate_decoder", "shor_bp_comm"]` | 1.705 ms (5%) | | 1.77 MiB (1%) | 42553 | | `["ecc", "evaluate_decoder", "shor_bp_naivesyn"]` | 3.787 ms (5%) | | 3.61 MiB (1%) | 81634 | | `["ecc", "evaluate_decoder", "shor_bp_shorsyn"]` | 4.164 ms (5%) | | 3.70 MiB (1%) | 82636 | | `["ecc", "evaluate_decoder", "shor_pybp_comm"]` | 64.280 ms (5%) | | 4.76 MiB (1%) | 144207 | | `["ecc", "evaluate_decoder", "shor_pybp_naivesyn"]` | 133.041 ms (5%) | | 9.57 MiB (1%) | 284018 | | `["ecc", "evaluate_decoder", "shor_pybp_shorsyn"]` | 129.201 ms (5%) | | 9.66 MiB (1%) | 285104 | | `["ecc", "evaluate_decoder", "shor_table_comm"]` | 241.610 μs (5%) | | 241.42 KiB (1%) | 4593 | | `["ecc", "evaluate_decoder", "shor_table_naivesyn"]` | 798.937 μs (5%) | | 334.20 KiB (1%) | 4790 | | `["ecc", "evaluate_decoder", "shor_table_shorsyn"]` | 1.172 ms (5%) | | 427.73 KiB (1%) | 5876 | | `["ecc", "evaluate_decoder", "toric8_bp_comm"]` | 640.512 ms (5%) | 10.718 ms | 166.49 MiB (1%) | 998936 | | `["ecc", "evaluate_decoder", "toric8_bp_naivesyn"]` | 1.387 s (5%) | 24.453 ms | 351.19 MiB (1%) | 2127202 | | `["ecc", "evaluate_decoder", "toric8_bp_shorsyn"]` | 1.386 s (5%) | 25.639 ms | 354.44 MiB (1%) | 2163963 | | `["ecc", "evaluate_decoder", "toric8_pybp_comm"]` | 457.341 ms (5%) | | 8.11 MiB (1%) | 163696 | | `["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"]` | 927.357 ms (5%) | | 18.04 MiB (1%) | 360388 | | `["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"]` | 955.391 ms (5%) | | 20.56 MiB (1%) | 392802 | | `["ecc", "evaluate_decoder", "toric8_pymatch_comm"]` | 3.883 ms (5%) | | 1.95 MiB (1%) | 23176 | | `["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"]` | 11.745 ms (5%) | | 5.58 MiB (1%) | 79355 | | `["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"]` | 19.647 ms (5%) | | 8.09 MiB (1%) | 111769 | | `["ecc", "evaluate_decoder", "toric8_table_comm"]` | 2.454 ms (5%) | | 1.64 MiB (1%) | 23082 | | `["ecc", "evaluate_decoder", "toric8_table_naivesyn"]` | 9.399 ms (5%) | | 4.93 MiB (1%) | 79160 | | `["ecc", "evaluate_decoder", "toric8_table_shorsyn"]` | 17.538 ms (5%) | | 7.44 MiB (1%) | 111574 | | `["pauli", "mul", "100"]` | 13.562 ns (5%) | | | | | `["pauli", "mul", "1000"]` | 17.344 ns (5%) | | | | | `["pauli", "mul", "100000"]` | 681.267 ns (5%) | | | | | `["pauli", "mul", "20000000"]` | 163.234 μs (5%) | | | | | `["stabilizer", "canon", "cano500"]` | 2.887 ms (5%) | | | | | `["stabilizer", "canon", "diag_cano500"]` | 694.102 μs (5%) | | | | | `["stabilizer", "canon", "diag_gott500"]` | 3.778 ms (5%) | | 5.19 MiB (1%) | 34521 | | `["stabilizer", "canon", "diag_rref500"]` | 665.559 μs (5%) | | | | | `["stabilizer", "canon", "gott500"]` | 6.071 ms (5%) | | 5.19 MiB (1%) | 34531 | | `["stabilizer", "canon", "md_cano500"]` | 1.504 ms (5%) | | | | | `["stabilizer", "canon", "md_rref500"]` | 1.493 ms (5%) | | | | | `["stabilizer", "canon", "rref500"]` | 2.939 ms (5%) | | | | | `["stabilizer", "project", "destabilizer"]` | 16.992 μs (5%) | | 288 bytes (1%) | 5 | | `["stabilizer", "project", "stabilizer"]` | 8.897 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "tensor", "diag_pow5_20"]` | 1.080 ms (5%) | | 23.97 MiB (1%) | 34 | | `["stabilizer", "tensor", "pow5_20"]` | 2.679 μs (5%) | | 6.44 KiB (1%) | 31 | | `["stabilizer", "trace", "destabilizer"]` | 30.887 μs (5%) | | 80 bytes (1%) | 2 | | `["stabilizer", "trace", "stabilizer"]` | 22.552 μs (5%) | | 112 bytes (1%) | 3 | ## Benchmark Group List Here's a list of all the benchmark groups executed by this job: - `["circuitsim", "mctrajectories"]` - `["circuitsim", "mctrajectories_sumtype"]` - `["circuitsim", "mctrajectories_union"]` - `["circuitsim", "pftrajectories"]` - `["circuitsim", "pftrajectories_sumtype"]` - `["circuitsim", "pftrajectories_union"]` - `["clifford", "dense"]` - `["clifford", "symbolic"]` - `["ecc", "evaluate_decoder"]` - `["pauli", "mul"]` - `["stabilizer", "canon"]` - `["stabilizer", "project"]` - `["stabilizer", "tensor"]` - `["stabilizer", "trace"]` ## Julia versioninfo ``` Julia Version 1.12.0-DEV.334 Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Linux (x86_64-linux-gnu) Ubuntu 22.04.4 LTS uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64 CPU: AMD EPYC 7763 64-Core Processor: speed user nice sys idle irq #1 0 MHz 2543 s 0 s 272 s 5972 s 0 s #2 0 MHz 2933 s 0 s 284 s 5595 s 0 s #3 0 MHz 2016 s 0 s 298 s 6477 s 0 s #4 0 MHz 1869 s 0 s 293 s 6632 s 0 s Memory: 15.606498718261719 GB (13355.90625 MB free) Uptime: 885.05 sec Load Avg: 1.08 1.09 0.76 WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 4 virtu...*[Comment body truncated]*
Krastanov commented 5 months ago

@hongyehu , there are some failures related to the expectation value (the tests in test_nonclifford_quantumoptics.jl). I will not have the time to investigate today, but it seems I have messed up a convention somewhere and we are getting expectation values that are occasionally imaginary. Hopefully I will get to it early next week, but let me know if you would like to discuss it together if you have time to investigate it.

hongyehu commented 5 months ago

@Krastanov Hi Stefan, I can take a look into the expect! function and see whether we missed anything!

Krastanov commented 5 months ago

here is a small reproducers:

using QuantumClifford
using QuantumOpticsBase # which provides Operator and Ket for conversion (kinda like qutip)
using LinearAlgebra

qo_basis = SpinBasis(1//2)
qo_tgate = sparse(identityoperator(qo_basis))
qo_tgate.data[2,2] = exp(im*pi/4)

for s in [S"X", S"Y", S"Z", S"-X", S"-Y", S"-Z"]
    for p in [P"X", P"Y", P"Z", P"-X", P"-Y", P"-Z"]
        gs = GeneralizedStabilizer(s)
        apply!(gs, pcT)
        ρ = dm(qo_tgate*Ket(s))
        @test Operator(gs) ≈ ρ
        if isapprox(expect(p, gs), expect(Operator(p),ρ); atol=1e-5)
        else
            println("failure of expectation calculation for state = T*(", s, ") and observable = ", p)
        end
    end
end

It prints the following failing cases:

failure of expect for state = T*(+ X) and observable = + Y
failure of expect for state = T*(+ X) and observable = - Y
failure of expect for state = T*(+ Y) and observable = + X
failure of expect for state = T*(+ Y) and observable = - X
failure of expect for state = T*(- X) and observable = + Y
failure of expect for state = T*(- X) and observable = - Y
failure of expect for state = T*(- Y) and observable = + X
failure of expect for state = T*(- Y) and observable = - X
Fe-r-oz commented 2 months ago

Hello, @Krastanov, I adressed some of the TODOs. Please do let me know do these #TODO improvements look plausible.

#After TODO 1: consider using bitpacking and SIMD xor with less eager shortcircuiting -- probably would be much faster

"""Same as `all(==(0), (a.+b.+c) .% 2)`"""
function _allthreesumtozero(a, b, c)
    packed_sum = a .+ b .+ c
    all_even = all(@. iseven(packed_sum))
    return all_even
end

After TODO 2: there must be a cleaner way to do this

function _dictvaltype(dict)
    return Base.getindex(eltype(dict).parameters, 2)
end

----------------
example: suppose genstab
julia> e = zero(_dictvaltype(genstab.destabweights))
0.0 + 0.0im
-----------------

After  TODO 3:  newdict = typeof(dict)(tzero) # TODO jeez, this is ugly

function apply!(state::GeneralizedStabilizer, gate::PauliChannel)
    dict = state.destabweights
    stab = state.stab
    dtype = _dictvaltype(dict)
    tzero = zero(dtype)
    tone = one(dtype)
    newdict = DefaultDict{Tuple{BitVector, BitVector}, Complex{Float64}}(Complex{Float64}(0, 0))
    for ((dᵢ,dⱼ), χ) in dict
        for ((Pₗ,Pᵣ), w) in zip(gate.paulis, gate.weights)
            phaseₗ, dₗ, dₗˢᵗᵃᵇ = rowdecompose(Pₗ, stab)
            phaseᵣ, dᵣ, dᵣˢᵗᵃᵇ = rowdecompose(Pᵣ, stab)
            c = (dot(dₗˢᵗᵃᵇ, dᵢ) + dot(dᵣˢᵗᵃᵇ, dⱼ)) * 2
            dᵢ′ = dₗ .⊻ dᵢ
            dⱼ′ = dᵣ .⊻ dⱼ
            χ′ = χ * w * (-tone)^c * (im)^(-phaseₗ + phaseᵣ + 4)
            newdict[(dᵢ′, dⱼ′)] += χ′
        end
    end
    keys_to_delete = Tuple[]
    for (k, v) in newdict
        if abs(v) < 1e-14
            push!(keys_to_delete, k)
        end
    end
    for k in keys_to_delete
        delete!(newdict, k)
    end
    state.destabweights = newdict
    return state
end

I reran the this test to confirm that there was no error:

julia> for n in 1:5
           i = rand(1:n)
           stab = random_stabilizer(n)
           genstab = GeneralizedStabilizer(stab)
           ket = Ket(stab)
           @test dm(ket) ≈ Operator(stab)
           @test dm(ket) ≈ Operator(genstab)

           pauli = random_pauli(n; nophase=false, realphase=true)
           qo_pauli = Operator(pauli)

           qo_bigtgate = n==1 ? qo_tgate : embed(qo_basis^n, i, qo_tgate)
           bigtgate = embed(n,i, pcT)
           @test qo_bigtgate ≈ Operator(bigtgate)

           for step in 1:10
               # apply!(ket, qo_bigtgate) TODO implement this API
               ket = qo_bigtgate*ket
               apply!(genstab, bigtgate)
               @test dm(ket) ≈ Operator(genstab)
               @test isapprox(expect(qo_pauli, ket), expect(pauli, genstab); atol=1e-10)
           end
       end
Krastanov commented 2 months ago

Hi, Feroz. I am not following what you mean with your last comment. If you want to discus some changes to the code, I would suggest making a new branch or even a pull request on which we can talk. Very importantly, though, if you want to discuss specific changes, make sure you have clear separation of a single semantic change per commit. You will need to learn how to use an interactive rebase and potentially do force pushes, to keep the history of your changes easy to work with.