add performance tweaks, primarily addresser-state caching, and some s…

[ghost]

…afety

src/addresser/rewiring.lisp

• change make-rewiring to not cons needlessly

src/compilation-methods.lisp src/addresser/addresser-state.lisp src/addresser/fidelity-addresser.lisp addresser/temporal-addresser.lisp src/chip/chip-specification.lisp

• have a global addresser-state cache that compiler-hook uses
• comment about safety in warm-up-addresser-state, namely, why it's safe NOT to unwind-protect code that restores slots to addresser state

[braised-babbage]

Re: the tests failing, this is because of the changes I made in https://github.com/quil-lang/quilc/pull/755 (specifically https://github.com/quil-lang/quilc/commit/e8d8c6c363a80d104a5775d28588e4934602f062 bumping the magicl dependency to a version not yet released on quicklisp). The CI uses quicklisp to resolve these dependencies -- if instead we were pulling in the latest git branches or appropriate git tags or something, they would pass.

[ghost]

(last couple of pushes were minor amends to commit message)

[ghost]

Here's benchmarking output for make benchmark-nq-2x in this branch:

+----------------------------+----------+-------+----------+
|        NAME                | TIME (s) | SWAPS | 2Q DEPTH |
+----------------------------+----------+-------+----------+
| STATIC/FULLY-CONNECTED 10  |     0.46 |     0 |        2 |
| STATIC/FULLY-CONNECTED 30  |     0.37 |     0 |        2 |
| STATIC/FULLY-CONNECTED 50  |     0.39 |     0 |        2 |
| STATIC/FULLY-CONNECTED 70  |     0.51 |     0 |        2 |
+----------------------------+----------+-------+----------+
+----------------------------+----------+-------+----------+
|        NAME                | TIME (s) | SWAPS | 2Q DEPTH |
+----------------------------+----------+-------+----------+
| BELL/FULLY-CONNECTED 10    |     2.92 |     0 |        9 |
| BELL/FULLY-CONNECTED 30    |     9.95 |     0 |       29 |
| BELL/FULLY-CONNECTED 50    |    18.22 |     0 |       49 |
| BELL/FULLY-CONNECTED 70    |    28.22 |     0 |       69 |
+----------------------------+----------+-------+----------+
+----------------------------+----------+-------+----------+
|        NAME                | TIME (s) | SWAPS | 2Q DEPTH |
+----------------------------+----------+-------+----------+
| QFT/FULLY-CONNECTED 10     |     7.80 |     0 |       37 |
| QFT/FULLY-CONNECTED 30     |    76.56 |     0 |      117 |
| QFT/FULLY-CONNECTED 50     |   240.67 |     0 |      197 |
| QFT/FULLY-CONNECTED 70     |   513.86 |     0 |      277 |
+----------------------------+----------+-------+----------+
+----------------------------+----------+-------+----------+
|        NAME                | TIME (s) | SWAPS | 2Q DEPTH |
+----------------------------+----------+-------+----------+
| HADAMARD/FULLY-CONNECTED 10 |     0.02 |     0 |        0 |
| HADAMARD/FULLY-CONNECTED 30 |     0.17 |     0 |        0 |
| HADAMARD/FULLY-CONNECTED 50 |     0.44 |     0 |        0 |
| HADAMARD/FULLY-CONNECTED 70 |     0.92 |     0 |        0 |
+----------------------------+----------+-------+----------+
+----------------------------+----------+-------+----------+
|        NAME                | TIME (s) | SWAPS | 2Q DEPTH |
+----------------------------+----------+-------+----------+
| STATIC/LINEAR 10           |     0.36 |     0 |        2 |
| STATIC/LINEAR 30           |     0.38 |     0 |        2 |
| STATIC/LINEAR 50           |     0.39 |     0 |        2 |
| STATIC/LINEAR 70           |     0.42 |     0 |        2 |
+----------------------------+----------+-------+----------+
+----------------------------+----------+-------+----------+
|        NAME                | TIME (s) | SWAPS | 2Q DEPTH |
+----------------------------+----------+-------+----------+
| BELL/LINEAR 10             |     2.23 |     0 |        9 |
| BELL/LINEAR 30             |     7.71 |     0 |       29 |
| BELL/LINEAR 50             |    14.11 |     0 |       49 |
| BELL/LINEAR 70             |    21.76 |     0 |       69 |
+----------------------------+----------+-------+----------+
+----------------------------+----------+-------+----------+
|        NAME                | TIME (s) | SWAPS | 2Q DEPTH |
+----------------------------+----------+-------+----------+
| QFT/LINEAR 10              |     5.35 |    61 |       79 |
| QFT/LINEAR 30              |    16.91 |   641 |      265 |
| QFT/LINEAR 50              |    59.70 |  1821 |      457 |
| QFT/LINEAR 70              |   155.42 |  3599 |      637 |
+----------------------------+----------+-------+----------+
+----------------------------+----------+-------+----------+
|        NAME                | TIME (s) | SWAPS | 2Q DEPTH |
+----------------------------+----------+-------+----------+
| HADAMARD/LINEAR 10         |     0.02 |     0 |        0 |
| HADAMARD/LINEAR 30         |     0.16 |     0 |        0 |
| HADAMARD/LINEAR 50         |     0.43 |     0 |        0 |
| HADAMARD/LINEAR 70         |     0.94 |     0 |        0 |
+----------------------------+----------+-------+----------+

+----------------------------+----------+-------+----------+
|        NAME                | TIME (s) | SWAPS | 2Q DEPTH |
+----------------------------+----------+-------+----------+
| STATIC/FULLY-CONNECTED 10  |     0.06 |     0 |        2 |
| STATIC/FULLY-CONNECTED 30  |     0.07 |     0 |        2 |
| STATIC/FULLY-CONNECTED 50  |     0.08 |     0 |        2 |
| STATIC/FULLY-CONNECTED 70  |     0.12 |     0 |        2 |
+----------------------------+----------+-------+----------+
+----------------------------+----------+-------+----------+
|        NAME                | TIME (s) | SWAPS | 2Q DEPTH |
+----------------------------+----------+-------+----------+
| BELL/FULLY-CONNECTED 10    |     0.34 |     0 |        9 |
| BELL/FULLY-CONNECTED 30    |     1.50 |     0 |       29 |
| BELL/FULLY-CONNECTED 50    |     3.59 |     0 |       49 |
| BELL/FULLY-CONNECTED 70    |     7.65 |     0 |       69 |
+----------------------------+----------+-------+----------+
+----------------------------+----------+-------+----------+
|        NAME                | TIME (s) | SWAPS | 2Q DEPTH |
+----------------------------+----------+-------+----------+
| QFT/FULLY-CONNECTED 10     |     1.22 |     0 |       37 |
| QFT/FULLY-CONNECTED 30     |    18.52 |     0 |      117 |
| QFT/FULLY-CONNECTED 50     |    84.73 |     0 |      197 |
| QFT/FULLY-CONNECTED 70     |   268.54 |     0 |      277 |
+----------------------------+----------+-------+----------+
+----------------------------+----------+-------+----------+
|        NAME                | TIME (s) | SWAPS | 2Q DEPTH |
+----------------------------+----------+-------+----------+
| HADAMARD/FULLY-CONNECTED 10 |     0.02 |     0 |        0 |
| HADAMARD/FULLY-CONNECTED 30 |     0.17 |     0 |        0 |
| HADAMARD/FULLY-CONNECTED 50 |     0.43 |     0 |        0 |
| HADAMARD/FULLY-CONNECTED 70 |     0.91 |     0 |        0 |
+----------------------------+----------+-------+----------+
+----------------------------+----------+-------+----------+
|        NAME                | TIME (s) | SWAPS | 2Q DEPTH |
+----------------------------+----------+-------+----------+
| STATIC/LINEAR 10           |     0.07 |     0 |        2 |
| STATIC/LINEAR 30           |     0.08 |     0 |        2 |
| STATIC/LINEAR 50           |     0.10 |     0 |        2 |
| STATIC/LINEAR 70           |     0.12 |     0 |        2 |
+----------------------------+----------+-------+----------+
+----------------------------+----------+-------+----------+
|        NAME                | TIME (s) | SWAPS | 2Q DEPTH |
+----------------------------+----------+-------+----------+
| BELL/LINEAR 10             |     0.38 |     0 |        9 |
| BELL/LINEAR 30             |     1.60 |     0 |       29 |
| BELL/LINEAR 50             |     3.73 |     0 |       49 |
| BELL/LINEAR 70             |     7.02 |     0 |       69 |
+----------------------------+----------+-------+----------+
+----------------------------+----------+-------+----------+
|        NAME                | TIME (s) | SWAPS | 2Q DEPTH |
+----------------------------+----------+-------+----------+
| QFT/LINEAR 10              |     1.68 |    61 |       79 |
| QFT/LINEAR 30              |    16.80 |   641 |      265 |
| QFT/LINEAR 50              |    59.14 |  1821 |      457 |
| QFT/LINEAR 70              |   155.20 |  3601 |      637 |
+----------------------------+----------+-------+----------+
+----------------------------+----------+-------+----------+
|        NAME                | TIME (s) | SWAPS | 2Q DEPTH |
+----------------------------+----------+-------+----------+
| HADAMARD/LINEAR 10         |     0.03 |     0 |        0 |
| HADAMARD/LINEAR 30         |     0.17 |     0 |        0 |
| HADAMARD/LINEAR 50         |     0.43 |     0 |        0 |
| HADAMARD/LINEAR 70         |     0.90 |     0 |        0 |
+----------------------------+----------+-------+----------+

Run on 2021-10-27 MacBook Pro (16-16, 2019): macOS Catalina 10.15.7: Memory: 32 GB

[ghost]

Here are several charts showing benchmark data for select program/chip combinations in this branch vs. baseline (master).

hadamard_fully-connected:

hadamard_linear:

qft_fully-connected:

qft_linear:

[ghost]

Here's a CSV file of data corresponding to the above ASCII output tables and backing up the above charts. 2021-10-28-benchmark-nq-data.csv

[ghost]

FYI @kilimanjaro @stylewarning @notmgsk all the last several commits today were not really intended as part of this commit, but rather are part of an attempt to work out a different way to test cl-quil such that it gets the master version of qvm and magicl.

However, it's not really working out, since it's trying to do cd ../qvm, and the CI environment will not allow that. Any ideas?

[ghost]

So I tried going back to using the Dockerfile, namely

• undid all the 'git pull ...' shananigans in Makefile
• undid the stuff in the yaml file
• added versioning to Dockerfile for magicl ala qvm
• it got an error: (log:)

  Step 7/21 : FROM rigetti/magicl:$magicl_version as magicl
  pull access denied for rigetti/magicl, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
  Error: Process completed with exit code 1.

• so that doesn't work
• Seems someone with higher access/authorization (@stylewarning or @notmgsk ?) can probably get one of these approaches I've tried to work.

[stylewarning]

@mhdavid-hrl Can we move CI fixes into a separate branch and PR? It'll be just as good a test I think.

[ghost]

Yes, I will presently move CI stuff into a separate branch/PR and get this PR/branch to just embody the originally intended changes. EDIT: now done.

[ghost]

This CI commits discussed and pushed into this PR branch have now been spun off into a new (WIP) PR: "WIP: various Github CI improvements for quilc #768".

[ghost]

Latest force-push reset commits in this PR branch back to use weak hash table for addresser state cache AND included a rebase on master branch.

[stylewarning]

We are tabling this PR for now. See issue #769 for details.