Closed larseggert closed 1 week ago
All modified and coverable lines are covered by tests :white_check_mark:
Please upload report for BASE (
main@d6279bf
). Learn more about missing BASE report. Report is 1 commits behind head on main.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
QUIC Interop Runner, client vs. server
QUIC Interop Runner, client vs. server
QUIC Interop Runner, client vs. server
Performance differences relative to d6279bf40c91c4cb45c358d5f614dc6190e4c9eb.
time: [134.95 ns 135.99 ns 137.65 ns] change: [+35.233% +35.998% +36.851%] (p = 0.00 < 0.05) Found 6 outliers among 100 measurements (6.00%) 4 (4.00%) high mild 2 (2.00%) high severe
time: [165.25 ns 165.79 ns 166.33 ns] change: [+40.141% +40.797% +41.397%] (p = 0.00 < 0.05) Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe
time: [164.69 ns 165.26 ns 165.90 ns] change: [+39.722% +40.576% +41.307%] (p = 0.00 < 0.05) Found 3 outliers among 100 measurements (3.00%) 2 (2.00%) high mild 1 (1.00%) high severe
time: [138.28 ns 138.45 ns 138.63 ns] change: [+40.759% +41.880% +42.795%] (p = 0.00 < 0.05) Found 11 outliers among 100 measurements (11.00%) 4 (4.00%) high mild 7 (7.00%) high severe
time: [111.79 ms 111.84 ms 111.90 ms] change: [+0.5702% +0.6418% +0.7122%] (p = 0.00 < 0.05) Found 6 outliers among 100 measurements (6.00%) 6 (6.00%) low mild
time: [26.458 ms 27.705 ms 28.939 ms] change: [-7.3012% -1.5130% +4.9817%] (p = 0.64 > 0.05)
time: [36.467 ms 38.134 ms 39.796 ms] change: [-2.7432% +3.9825% +11.695%] (p = 0.27 > 0.05)
time: [26.027 ms 26.824 ms 27.622 ms] change: [-2.4103% +2.0917% +6.5536%] (p = 0.35 > 0.05) Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high mild
time: [41.834 ms 44.522 ms 47.228 ms] change: [-3.9072% +3.2513% +11.562%] (p = 0.41 > 0.05)
time: [112.65 ms 113.13 ms 113.59 ms] thrpt: [880.40 MiB/s 883.97 MiB/s 887.74 MiB/s] change: time: [-1.6310% -1.1234% -0.6488%] (p = 0.00 < 0.05) thrpt: [+0.6530% +1.1362% +1.6580%] Found 3 outliers among 100 measurements (3.00%) 3 (3.00%) low mild
time: [310.93 ms 314.62 ms 318.26 ms] thrpt: [31.420 Kelem/s 31.784 Kelem/s 32.161 Kelem/s] change: time: [-2.9109% -1.3456% +0.2944%] (p = 0.10 > 0.05) thrpt: [-0.2935% +1.3640% +2.9982%]
time: [33.735 ms 33.932 ms 34.141 ms] thrpt: [29.291 elem/s 29.471 elem/s 29.643 elem/s] change: time: [-1.9031% -1.0755% -0.2413%] (p = 0.01 < 0.05) thrpt: [+0.2419% +1.0872% +1.9400%] Found 5 outliers among 100 measurements (5.00%) 5 (5.00%) high mild
Transfer of 33554432 bytes over loopback. | Client | Server | CC | Pacing | Mean [ms] | Min [ms] | Max [ms] | Relative |
---|---|---|---|---|---|---|---|---|
msquic | msquic | 165.4 ± 114.8 | 90.3 | 562.6 | 1.00 | |||
neqo | msquic | reno | on | 256.6 ± 71.2 | 207.1 | 436.2 | 1.00 | |
neqo | msquic | reno | 294.8 ± 105.0 | 222.9 | 521.4 | 1.00 | ||
neqo | msquic | cubic | on | 278.2 ± 101.9 | 221.1 | 514.1 | 1.00 | |
neqo | msquic | cubic | 309.3 ± 101.4 | 219.4 | 506.6 | 1.00 | ||
msquic | neqo | reno | on | 138.9 ± 85.2 | 85.5 | 368.3 | 1.00 | |
msquic | neqo | reno | 149.8 ± 126.2 | 84.2 | 603.0 | 1.00 | ||
msquic | neqo | cubic | on | 181.5 ± 114.2 | 91.4 | 411.9 | 1.00 | |
msquic | neqo | cubic | 210.1 ± 190.6 | 86.2 | 976.8 | 1.00 | ||
neqo | neqo | reno | on | 163.7 ± 63.1 | 131.2 | 392.8 | 1.00 | |
neqo | neqo | reno | 180.7 ± 82.0 | 124.4 | 420.8 | 1.00 | ||
neqo | neqo | cubic | on | 213.2 ± 77.1 | 141.4 | 408.3 | 1.00 | |
neqo | neqo | cubic | 192.0 ± 63.2 | 138.5 | 354.6 | 1.00 |
So that we get issues for platform-specific code.
Also change
macos-14
->macos-latest
everywhere.