search

mozilla / neqo

Neqo, the Mozilla Firefox implementation of QUIC in Rust
https://firefox-source-docs.mozilla.org/networking/http/http3.html
Apache License 2.0
1.83k stars 123 forks source link

test(transport): check local and remote GREASE_QUIC_BIT TP #2101

Closed mxinden closed 2 weeks ago

mxinden commented 2 weeks ago

Follow-up to https://github.com/mozilla/neqo/pull/2100.

Test succeeds when run on top of https://github.com/mozilla/neqo/pull/2100.

@larseggert feel free to cherry-pick in case you find this useful.

github-actions[bot] commented 2 weeks ago

Failed Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

larseggert commented 2 weeks ago

Thanks, we can land this after #2100.

github-actions[bot] commented 2 weeks ago

Firefox builds for this PR

The following builds are available for testing. Crossed-out builds did not succeed.

codecov[bot] commented 2 weeks ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 95.38%. Comparing base (2ca29c1) to head (2a0492e). Report is 1 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2101 +/- ## ========================================== + Coverage 95.35% 95.38% +0.02% ========================================== Files 112 112 Lines 36325 36325 ========================================== + Hits 34638 34647 +9 + Misses 1687 1678 -9 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

github-actions[bot] commented 2 weeks ago

Benchmark results

Performance differences relative to 0b106c286f8f9c879f2122daab9e8bd12aa9a703.

coalesce_acked_from_zero 1+1 entries: No change in performance detected.
       time:   [99.351 ns 99.683 ns 100.02 ns]
       change: [-0.1542% +0.6098% +1.5340%] (p = 0.17 > 0.05)

Found 15 outliers among 100 measurements (15.00%)
  10 (10.00%) high mild
  5 (5.00%) high severe
coalesce_acked_from_zero 3+1 entries: No change in performance detected.
       time:   [117.23 ns 117.56 ns 117.92 ns]
       change: [-0.0962% +0.3521% +0.8000%] (p = 0.14 > 0.05)

Found 15 outliers among 100 measurements (15.00%)
  2 (2.00%) low mild
  13 (13.00%) high severe
coalesce_acked_from_zero 10+1 entries: No change in performance detected.
       time:   [116.98 ns 118.23 ns 120.34 ns]
       change: [+0.0235% +1.0363% +2.4624%] (p = 0.09 > 0.05)

Found 17 outliers among 100 measurements (17.00%)
  5 (5.00%) low severe
  1 (1.00%) low mild
  1 (1.00%) high mild
  10 (10.00%) high severe
coalesce_acked_from_zero 1000+1 entries: No change in performance detected.
       time:   [97.378 ns 97.571 ns 97.807 ns]
       change: [-1.0251% -0.1066% +0.8405%] (p = 0.83 > 0.05)

Found 12 outliers among 100 measurements (12.00%)
  3 (3.00%) high mild
  9 (9.00%) high severe
RxStreamOrderer::inbound_frame(): Change within noise threshold.
       time:   [112.47 ms 112.61 ms 112.83 ms]
       change: [+0.9661% +1.1095% +1.3312%] (p = 0.00 < 0.05)

Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) low mild
  1 (1.00%) high severe
transfer/pacing-false/varying-seeds: No change in performance detected.
       time:   [25.839 ms 26.842 ms 27.841 ms]
       change: [-9.2546% -4.2640% +0.6030%] (p = 0.10 > 0.05)
transfer/pacing-true/varying-seeds: No change in performance detected.
       time:   [34.514 ms 36.361 ms 38.253 ms]
       change: [-5.6946% +0.5461% +7.4262%] (p = 0.88 > 0.05)

Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
transfer/pacing-false/same-seed: No change in performance detected.
       time:   [31.305 ms 32.021 ms 32.727 ms]
       change: [-5.1059% -2.2080% +0.9687%] (p = 0.17 > 0.05)
transfer/pacing-true/same-seed: Change within noise threshold.
       time:   [38.192 ms 40.661 ms 43.139 ms]
       change: [-18.383% -10.608% -1.9496%] (p = 0.02 < 0.05)

Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) low mild
1-conn/1-100mb-resp (aka. Download)/client: Change within noise threshold.
       time:   [113.08 ms 113.54 ms 114.04 ms]
       thrpt:  [876.90 MiB/s 880.77 MiB/s 884.31 MiB/s]
change:
       time:   [-1.8711% -1.3187% -0.7524%] (p = 0.00 < 0.05)
       thrpt:  [+0.7581% +1.3363% +1.9068%]

Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) low mild
  1 (1.00%) high severe
1-conn/10_000-parallel-1b-resp (aka. RPS)/client: No change in performance detected.
       time:   [312.60 ms 316.45 ms 320.33 ms]
       thrpt:  [31.218 Kelem/s 31.600 Kelem/s 31.990 Kelem/s]
change:
       time:   [-2.2632% -0.6558% +1.0318%] (p = 0.45 > 0.05)
       thrpt:  [-1.0213% +0.6601% +2.3156%]

Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
1-conn/1-1b-resp (aka. HPS)/client: No change in performance detected.
       time:   [33.948 ms 34.124 ms 34.316 ms]
       thrpt:  [29.141  elem/s 29.305  elem/s 29.456  elem/s]
change:
       time:   [-0.9045% -0.0607% +0.7617%] (p = 0.89 > 0.05)
       thrpt:  [-0.7559% +0.0608% +0.9127%]

Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) high mild
  3 (3.00%) high severe

Client/server transfer results

Transfer of 33554432 bytes over loopback. Client Server CC Pacing Mean [ms] Min [ms] Max [ms] Relative
msquic msquic 201.3 ± 94.5 100.6 355.0 1.00
neqo msquic reno on 268.1 ± 84.7 214.0 422.0 1.00
neqo msquic reno 241.1 ± 56.1 208.7 414.4 1.00
neqo msquic cubic on 290.5 ± 96.5 217.7 436.4 1.00
neqo msquic cubic 253.4 ± 75.5 210.1 462.3 1.00
msquic neqo reno on 155.1 ± 86.3 94.5 323.0 1.00
msquic neqo reno 158.4 ± 82.2 95.0 323.7 1.00
msquic neqo cubic on 153.0 ± 93.9 93.3 447.0 1.00
msquic neqo cubic 180.0 ± 109.5 82.3 346.7 1.00
neqo neqo reno on 314.1 ± 123.4 129.8 440.3 1.00
neqo neqo reno 231.8 ± 112.2 139.8 430.2 1.00
neqo neqo cubic on 223.7 ± 104.2 140.3 424.8 1.00
neqo neqo cubic 270.5 ± 138.5 133.4 606.0 1.00

:arrow_down: Download logs