search

mozilla / neqo

Neqo, the Mozilla Firefox implementation of QUIC in Rust
https://firefox-source-docs.mozilla.org/networking/http/http3.html
Apache License 2.0
1.85k stars 124 forks source link

ci: Pass a PAT to rust/action.yml to avoid hitting download rate limits #2049

Closed larseggert closed 3 months ago

larseggert commented 3 months ago

See https://github.com/cargo-bins/cargo-quickinstall/issues/267#issuecomment-2277888539

github-actions[bot] commented 3 months ago

Failed Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

github-actions[bot] commented 3 months ago

Benchmark results

Performance differences relative to ca0080e41ec3b06f1813da1c774ca321231e7eec.

coalesce_acked_from_zero 1+1 entries: :broken_heart: Performance has regressed.
       time:   [199.88 ns 200.39 ns 200.93 ns]
       change: [+1.5580% +2.5395% +3.5841%] (p = 0.00 < 0.05)

Found 17 outliers among 100 measurements (17.00%)
  1 (1.00%) low severe
  3 (3.00%) low mild
  8 (8.00%) high mild
  5 (5.00%) high severe
coalesce_acked_from_zero 3+1 entries: :broken_heart: Performance has regressed.
       time:   [245.59 ns 246.23 ns 246.90 ns]
       change: [+3.6789% +4.0095% +4.3371%] (p = 0.00 < 0.05)

Found 19 outliers among 100 measurements (19.00%)
  2 (2.00%) high mild
  17 (17.00%) high severe
coalesce_acked_from_zero 10+1 entries: :broken_heart: Performance has regressed.
       time:   [244.60 ns 245.46 ns 246.43 ns]
       change: [+3.1276% +3.5517% +3.9396%] (p = 0.00 < 0.05)

Found 11 outliers among 100 measurements (11.00%)
  1 (1.00%) low mild
  4 (4.00%) high mild
  6 (6.00%) high severe
coalesce_acked_from_zero 1000+1 entries: :broken_heart: Performance has regressed.
       time:   [224.97 ns 230.58 ns 243.50 ns]
       change: [+3.4899% +6.5729% +11.983%] (p = 0.00 < 0.05)

Found 10 outliers among 100 measurements (10.00%)
  5 (5.00%) high mild
  5 (5.00%) high severe
RxStreamOrderer::inbound_frame(): :green_heart: Performance has improved.
       time:   [118.68 ms 118.74 ms 118.82 ms]
       change: [-2.2822% -2.2153% -2.1461%] (p = 0.00 < 0.05)

Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high mild
transfer/pacing-false/varying-seeds: No change in performance detected.
       time:   [40.165 ms 42.071 ms 43.963 ms]
       change: [-6.1017% +0.4981% +7.6114%] (p = 0.89 > 0.05)
transfer/pacing-true/varying-seeds: No change in performance detected.
       time:   [53.124 ms 56.116 ms 59.186 ms]
       change: [-11.069% -4.1953% +3.3351%] (p = 0.26 > 0.05)

Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
transfer/pacing-false/same-seed: No change in performance detected.
       time:   [48.691 ms 50.205 ms 51.701 ms]
       change: [-3.8448% +0.3942% +4.7958%] (p = 0.86 > 0.05)
transfer/pacing-true/same-seed: No change in performance detected.
       time:   [69.792 ms 76.447 ms 82.943 ms]
       change: [-8.9823% +2.9479% +16.485%] (p = 0.64 > 0.05)
1-conn/1-100mb-resp (aka. Download)/client: No change in performance detected.
       time:   [169.51 ms 171.24 ms 173.01 ms]
       thrpt:  [577.99 MiB/s 583.98 MiB/s 589.93 MiB/s]
change:
       time:   [-2.9784% -1.1953% +0.5418%] (p = 0.19 > 0.05)
       thrpt:  [-0.5389% +1.2098% +3.0698%]

Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
1-conn/10_000-parallel-1b-resp (aka. RPS)/client: No change in performance detected.
       time:   [406.55 ms 410.14 ms 413.71 ms]
       thrpt:  [24.171 Kelem/s 24.382 Kelem/s 24.597 Kelem/s]
change:
       time:   [-0.7539% +0.4575% +1.5888%] (p = 0.45 > 0.05)
       thrpt:  [-1.5639% -0.4554% +0.7597%]
1-conn/1-1b-resp (aka. HPS)/client: No change in performance detected.
       time:   [45.763 ms 46.444 ms 47.128 ms]
       thrpt:  [21.219  elem/s 21.531  elem/s 21.852  elem/s]
change:
       time:   [-3.3723% -1.2180% +0.8088%] (p = 0.27 > 0.05)
       thrpt:  [-0.8023% +1.2331% +3.4900%]

Client/server transfer results

Transfer of 33554432 bytes over loopback. Client Server CC Pacing Mean [ms] Min [ms] Max [ms] Relative
msquic msquic 233.8 ± 119.4 109.1 529.9 1.00
neqo msquic reno on 291.0 ± 59.6 258.6 459.0 1.00
neqo msquic reno 343.3 ± 87.8 259.5 471.1 1.00
neqo msquic cubic on 293.7 ± 59.3 256.7 457.1 1.00
neqo msquic cubic 317.0 ± 70.5 252.9 462.5 1.00
msquic neqo reno on 169.8 ± 68.7 115.9 324.3 1.00
msquic neqo reno 209.2 ± 77.0 137.5 358.1 1.00
msquic neqo cubic on 239.1 ± 87.6 147.0 395.2 1.00
msquic neqo cubic 238.6 ± 78.5 165.9 397.9 1.00
neqo neqo reno on 225.2 ± 107.4 151.0 501.9 1.00
neqo neqo reno 254.2 ± 96.2 154.0 409.4 1.00
neqo neqo cubic on 212.7 ± 84.8 154.8 369.0 1.00
neqo neqo cubic 198.6 ± 58.4 165.8 362.5 1.00

:arrow_down: Download logs

github-actions[bot] commented 3 months ago

Firefox builds for this PR

The following builds are available for testing. Crossed-out builds did not succeed.