Fix page load delays caused by API auth verification

[lo5]

v0.18, on examples/counter_unicast.py:

Baseline: 500rps, 30s

$ cat attack.txt | vegeta attack -duration=30s -rate=500 | tee results.bin | vegeta report
Requests      [total, rate, throughput]  15000, 500.03, 500.01
Duration      [total, attack, wait]      29.999543772s, 29.998173312s, 1.37046ms
Latencies     [mean, 50, 95, 99, max]    43.646089ms, 5.01603ms, 270.952433ms, 463.433687ms, 580.247216ms
Bytes In      [total, mean]              0, 0.00
Bytes Out     [total, mean]              285000, 19.00
Success       [ratio]                    100.00%
Status Codes  [code:count]               200:15000
Error Set:

Fulfillment:
  First request
    2021/10/25 09:03:45 * /None {"d":[{"k":"example items 0 button label","v":"Count=9"}]}
  Last request
    2021/10/25 09:06:16 * /None {"d":[{"k":"example items 0 button label","v":"Count=14930"}]}
  Elapsed time
    02:31

With HTTP Basic Auth disabled:

$ cat attack.txt | vegeta attack -duration=30s -rate=500 | tee results.bin | vegeta report
Requests      [total, rate, throughput]  15000, 500.03, 500.01
Duration      [total, attack, wait]      29.999317765s, 29.998388779s, 928.986µs
Latencies     [mean, 50, 95, 99, max]    1.893424ms, 855.601µs, 6.830168ms, 9.922392ms, 34.859879ms
Bytes In      [total, mean]              0, 0.00
Bytes Out     [total, mean]              285000, 19.00
Success       [ratio]                    100.00%
Status Codes  [code:count]               200:15000
Error Set:

Fulfillment:
  First request
    2021/10/25 10:30:21 * /None {"d":[{"k":"example items 0 button label","v":"Count=2"}]}
  Last request
    2021/10/25 10:30:51 * /None {"d":[{"k":"example items 0 button label","v":"Count=15000"}]}
  Elapsed time
    00:30

cc @supertramp01, @senalw

[lo5]

Added LRU cache to auth verification.

Results:

$ cat attack.txt | vegeta attack -duration=30s -rate=500 | tee results.bin | vegeta report
Requests      [total, rate, throughput]  15000, 500.03, 500.01
Duration      [total, attack, wait]      29.999262982s, 29.998363484s, 899.498µs
Latencies     [mean, 50, 95, 99, max]    1.887598ms, 823.374µs, 7.509762ms, 12.64735ms, 37.085127ms
Bytes In      [total, mean]              0, 0.00
Bytes Out     [total, mean]              285000, 19.00
Success       [ratio]                    100.00%
Status Codes  [code:count]               200:15000
Error Set:

[lo5]

Final results after https://github.com/h2oai/wave/issues/464

500rps for 30s

$ cat attack.txt | vegeta attack -duration=30s -rate=500 | tee results.bin | vegeta report
Requests      [total, rate, throughput]  15000, 500.03, 500.02
Duration      [total, attack, wait]      29.998930102s, 29.998182442s, 747.66µs
Latencies     [mean, 50, 95, 99, max]    1.335957ms, 760.765µs, 4.653907ms, 9.074042ms, 29.385884ms
Bytes In      [total, mean]              0, 0.00
Bytes Out     [total, mean]              285000, 19.00
Success       [ratio]                    100.00%
Status Codes  [code:count]               200:15000
Error Set:

750rps for 10s

elp@ugh:~/git/wave (master)
$ cat attack.txt | vegeta attack -duration=10s -rate=750 | tee results.bin | vegeta report
Requests      [total, rate, throughput]  7500, 750.16, 748.99
Duration      [total, attack, wait]      10.013486689s, 9.997928032s, 15.558657ms
Latencies     [mean, 50, 95, 99, max]    38.141352ms, 13.133795ms, 159.071975ms, 225.06916ms, 350.207037ms
Bytes In      [total, mean]              0, 0.00
Bytes Out     [total, mean]              142500, 19.00
Success       [ratio]                    100.00%
Status Codes  [code:count]               200:7500
Error Set:

750rps for 30s

$ cat attack.txt | vegeta attack -duration=30s -rate=750 | tee results.bin | vegeta report
Requests      [total, rate, throughput]  22500, 750.04, 745.08
Duration      [total, attack, wait]      30.198263689s, 29.998250496s, 200.013193ms
Latencies     [mean, 50, 95, 99, max]    111.416987ms, 12.266147ms, 520.635091ms, 724.367786ms, 905.216595ms
Bytes In      [total, mean]              0, 0.00
Bytes Out     [total, mean]              427500, 19.00
Success       [ratio]                    100.00%
Status Codes  [code:count]               200:22500
Error Set:

System (my desktop is a rusty old Intel NUC, you should get better results on newer machines):

Kernel: 5.13.0-7614-generic
CPU: Intel i7-6770HQ (8) @ 3.500GHz
GPU: Intel Iris Pro Graphics 580
Memory: 31996MiB