search

the-guild-org / gateways-benchmark

MIT License
43 stars 7 forks source link

Update dependency vega-lite to v5.21.0 #462

Closed renovate[bot] closed 1 day ago

renovate[bot] commented 1 day ago

This PR contains the following updates:

Package Change Age Adoption Passing Confidence
vega-lite (source) 5.17.0 -> 5.21.0 age adoption passing confidence

Release Notes

vega/vega-lite (vega-lite) ### [`v5.21.0`](https://redirect.github.com/vega/vega-lite/blob/HEAD/CHANGELOG.md#5210-2024-08-28) [Compare Source](https://redirect.github.com/vega/vega-lite/compare/v5.20.1...v5.21.0) ##### Bug Fixes - **[#​7724](https://redirect.github.com/vega/vega-lite/issues/7724),[#​9414](https://redirect.github.com/vega/vega-lite/issues/9414):** Make ticks respect bin/timeUnit bands and custom band ([#​9415](https://redirect.github.com/vega/vega-lite/issues/9415)) ([088094e](https://redirect.github.com/vega/vega-lite/commit/088094ee4401ca2c888a44057789093a2cd7f3df)), closes [#​7724](https://redirect.github.com/vega/vega-lite/issues/7724) [#​9414](https://redirect.github.com/vega/vega-lite/issues/9414) - make 1D tick mark responsive to width/height ([#​9411](https://redirect.github.com/vega/vega-lite/issues/9411)) ([c0c7b6b](https://redirect.github.com/vega/vega-lite/commit/c0c7b6b4ef126e138b48990366b24ae7d713dafe)) ##### Features - update code to not need implicit any, add hasProperty util ([#​9394](https://redirect.github.com/vega/vega-lite/issues/9394)) ([6ce344e](https://redirect.github.com/vega/vega-lite/commit/6ce344e3d3e92023ca991267565e39e195241485)) #### [5.20.1](https://redirect.github.com/vega/vega-lite/compare/v5.20.0...v5.20.1) (2024-07-31) ##### Bug Fixes - **[#​9401](https://redirect.github.com/vega/vega-lite/issues/9401):** conditional opacity should respect default value ([#​9402](https://redirect.github.com/vega/vega-lite/issues/9402)) ([b4c7e14](https://redirect.github.com/vega/vega-lite/commit/b4c7e1435339cd57cef620459ad6481aa4ec526e)), closes [#​9401](https://redirect.github.com/vega/vega-lite/issues/9401) ### [`v5.20.1`](https://redirect.github.com/vega/vega-lite/blob/HEAD/CHANGELOG.md#5201-2024-07-31) [Compare Source](https://redirect.github.com/vega/vega-lite/compare/v5.20.0...v5.20.1) ##### Bug Fixes - **[#​9401](https://redirect.github.com/vega/vega-lite/issues/9401):** conditional opacity should respect default value ([#​9402](https://redirect.github.com/vega/vega-lite/issues/9402)) ([b4c7e14](https://redirect.github.com/vega/vega-lite/commit/b4c7e1435339cd57cef620459ad6481aa4ec526e)), closes [#​9401](https://redirect.github.com/vega/vega-lite/issues/9401) ### [`v5.20.0`](https://redirect.github.com/vega/vega-lite/blob/HEAD/CHANGELOG.md#5200-2024-07-30) [Compare Source](https://redirect.github.com/vega/vega-lite/compare/v5.19.0...v5.20.0) ##### Bug Fixes - for issue when there are multiple aggregates with the same data field and aggregation operation ([#​9320](https://redirect.github.com/vega/vega-lite/issues/9320)) ([da84469](https://redirect.github.com/vega/vega-lite/commit/da844691cb4ad6e4d6cf32af90c7f14d248b4764)), closes [#​6184](https://redirect.github.com/vega/vega-lite/issues/6184) - make tick size responsive to plot height ([#​9395](https://redirect.github.com/vega/vega-lite/issues/9395)) ([3f5dce6](https://redirect.github.com/vega/vega-lite/commit/3f5dce6ed3df7783c61afd5fb80eb4520668aa04)) ##### Features - Expose `react` option for params ([#​9374](https://redirect.github.com/vega/vega-lite/issues/9374)) ([93d92ef](https://redirect.github.com/vega/vega-lite/commit/93d92efa4e1893fb01b77f8145ca3bf7bf8dba05)), closes [/github.com/vega/altair/pull/3394#issuecomment-2140320905](https://redirect.github.com//github.com/vega/altair/pull/3394/issues/issuecomment-2140320905) ### [`v5.19.0`](https://redirect.github.com/vega/vega-lite/blob/HEAD/CHANGELOG.md#5190-2024-06-14) [Compare Source](https://redirect.github.com/vega/vega-lite/compare/v5.18.1...v5.19.0) ##### Bug Fixes - **[#​3488](https://redirect.github.com/vega/vega-lite/issues/3488):** don't use zero as min for size scale by default so no points will be ever hidden ([#​9347](https://redirect.github.com/vega/vega-lite/issues/9347)) ([7db657a](https://redirect.github.com/vega/vega-lite/commit/7db657a23fca5ee849320e3ee8f3dcc13998f6a6)), closes [#​3488](https://redirect.github.com/vega/vega-lite/issues/3488) [#​3488](https://redirect.github.com/vega/vega-lite/issues/3488) - **[#​9343](https://redirect.github.com/vega/vega-lite/issues/9343):** binned stacked area now correctly filter null values ([#​9344](https://redirect.github.com/vega/vega-lite/issues/9344)) ([7cd3829](https://redirect.github.com/vega/vega-lite/commit/7cd3829dc2b297be5f954d2f6c685541435f8776)), closes [#​9343](https://redirect.github.com/vega/vega-lite/issues/9343) - fix typo in time unit 'utcweekdayhours' ([b32c73c](https://redirect.github.com/vega/vega-lite/commit/b32c73caf650a31319e634d3d98cc3c911e46031)) - use array for single item valueRefs in conditional if test exists ([#​9361](https://redirect.github.com/vega/vega-lite/issues/9361)) ([64201d6](https://redirect.github.com/vega/vega-lite/commit/64201d6d2a53364e6c214b07950b6ce77d174d5c)), closes [/github.com/vega/vega-lite/pull/9342/files#diff-f26cb911cd80159bbd659afb430b97fa7227afb4cc15e43565f2ca74c3361d22R64](https://redirect.github.com//github.com/vega/vega-lite/pull/9342/files/issues/diff-f26cb911cd80159bbd659afb430b97fa7227afb4cc15e43565f2ca74c3361d22R64) ##### Features - more options and examples for how marks and scales represent invalid data (e.g., nulls, NaNs) ([#​9342](https://redirect.github.com/vega/vega-lite/issues/9342)) ([502db1d](https://redirect.github.com/vega/vega-lite/commit/502db1d848ea0ea93a90d45995ad50fea353950a)), closes [#​8254](https://redirect.github.com/vega/vega-lite/issues/8254) #### [5.18.1](https://redirect.github.com/vega/vega-lite/compare/v5.18.0...v5.18.1) (2024-05-07) ##### Bug Fixes - Allow non-linear scale for stacked bars ([#​9315](https://redirect.github.com/vega/vega-lite/issues/9315)) ([716fe73](https://redirect.github.com/vega/vega-lite/commit/716fe734df2b4bd00fe78d7789ac43f5b035e4ed)) - correctly place unfiltered null values for line ([#​9326](https://redirect.github.com/vega/vega-lite/issues/9326)) ([#​9334](https://redirect.github.com/vega/vega-lite/issues/9334)) ([b617c84](https://redirect.github.com/vega/vega-lite/commit/b617c84d6fca636ff1469b7db5a40e7393354e5c)) - extent name collision in transforms ([#​9312](https://redirect.github.com/vega/vega-lite/issues/9312)) ([8b3b30b](https://redirect.github.com/vega/vega-lite/commit/8b3b30b352ce0e50fe5422cd4d24323a5a5ec9b4)) ### [`v5.18.1`](https://redirect.github.com/vega/vega-lite/blob/HEAD/CHANGELOG.md#5181-2024-05-07) [Compare Source](https://redirect.github.com/vega/vega-lite/compare/v5.18.0...v5.18.1) ##### Bug Fixes - Allow non-linear scale for stacked bars ([#​9315](https://redirect.github.com/vega/vega-lite/issues/9315)) ([716fe73](https://redirect.github.com/vega/vega-lite/commit/716fe734df2b4bd00fe78d7789ac43f5b035e4ed)) - correctly place unfiltered null values for line ([#​9326](https://redirect.github.com/vega/vega-lite/issues/9326)) ([#​9334](https://redirect.github.com/vega/vega-lite/issues/9334)) ([b617c84](https://redirect.github.com/vega/vega-lite/commit/b617c84d6fca636ff1469b7db5a40e7393354e5c)) - extent name collision in transforms ([#​9312](https://redirect.github.com/vega/vega-lite/issues/9312)) ([8b3b30b](https://redirect.github.com/vega/vega-lite/commit/8b3b30b352ce0e50fe5422cd4d24323a5a5ec9b4)) ### [`v5.18.0`](https://redirect.github.com/vega/vega-lite/blob/HEAD/CHANGELOG.md#5180-2024-04-09) [Compare Source](https://redirect.github.com/vega/vega-lite/compare/v5.17.0...v5.18.0) ##### Bug Fixes - **[#​8338](https://redirect.github.com/vega/vega-lite/issues/8338),[#​8126](https://redirect.github.com/vega/vega-lite/issues/8126):** make boxplot work with single value per group ([#​8339](https://redirect.github.com/vega/vega-lite/issues/8339)) ([4e55836](https://redirect.github.com/vega/vega-lite/commit/4e558369eecc5d84fbccd0fd2bc84be19f5f5e35)), closes [#​8338](https://redirect.github.com/vega/vega-lite/issues/8338) [#​8126](https://redirect.github.com/vega/vega-lite/issues/8126) [#​8338](https://redirect.github.com/vega/vega-lite/issues/8338) [#​8126](https://redirect.github.com/vega/vega-lite/issues/8126) - describe how to set a custom scheme by setting the range, reverts [#​9262](https://redirect.github.com/vega/vega-lite/issues/9262) ([#​9266](https://redirect.github.com/vega/vega-lite/issues/9266)) ([e6af641](https://redirect.github.com/vega/vega-lite/commit/e6af6414431ade1f52359ad682ffc1f4f4a667c3)), closes [#​9022](https://redirect.github.com/vega/vega-lite/issues/9022) - generate unescaped boxplot calculated field aliases ([#​9284](https://redirect.github.com/vega/vega-lite/issues/9284)) ([44fbceb](https://redirect.github.com/vega/vega-lite/commit/44fbceb0120ee54a8989cb1537537757adac8c58)) ##### Features - add explicit option to control how densities are resolved, change how densities are resolved by default ([#​9172](https://redirect.github.com/vega/vega-lite/issues/9172)) ([bf0b8d3](https://redirect.github.com/vega/vega-lite/commit/bf0b8d3e2c80dc1d28cd3c6c4e764ea2ed714d6c))

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

This PR was generated by Mend Renovate. View the repository job log.

github-actions[bot] commented 1 day ago

Overview for: federation-v1/constant-vus-over-time

This scenario runs 4 subgraphs and a GraphQL gateway with Federation v1 spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 100 VUs over 30s

Comparison

Comparison
Gateway RPS ⬇️ Requests Duration Notes
cosmo 193 5875 total, 0 failed avg: 332ms, p95: 719ms ✅
apollo-router 175 5350 total, 0 failed avg: 498ms, p95: 669ms ✅
wundergraph 153 4640 total, 0 failed avg: 529ms, p95: 946ms ✅
mesh-supergraph-bun 78 2411 total, 0 failed avg: 1239ms, p95: 2158ms ✅
apollo-server 52 1649 total, 0 failed avg: 1852ms, p95: 2294ms ✅
mercurius 28 890 total, 0 failed avg: 3406ms, p95: 4771ms ✅
mesh-supergraph 24 778 total, 0 failed avg: 4031ms, p95: 6411ms ✅
Summary for: `cosmo` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 17625 ✗ 0 data_received..................: 516 MB 17 MB/s data_sent......................: 7.0 MB 230 kB/s http_req_blocked...............: avg=338.27µs min=1.44µs med=2.8µs max=592.68ms p(90)=4.07µs p(95)=5.71µs http_req_connecting............: avg=262.82µs min=0s med=0s max=592.62ms p(90)=0s p(95)=0s http_req_duration..............: avg=332.17ms min=6.12ms med=293.95ms max=1.77s p(90)=600.55ms p(95)=718.51ms { expected_response:true }...: avg=332.17ms min=6.12ms med=293.95ms max=1.77s p(90)=600.55ms p(95)=718.51ms http_req_failed................: 0.00% ✓ 0 ✗ 5875 http_req_receiving.............: avg=81.92ms min=34.48µs med=77.71µs max=1.58s p(90)=360.73ms p(95)=494.12ms http_req_sending...............: avg=4.94ms min=7.98µs med=13.37µs max=1.06s p(90)=102.5µs p(95)=10ms http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=245.3ms min=6.02ms med=228.87ms max=748.31ms p(90)=433.77ms p(95)=491.53ms http_reqs......................: 5875 193.744005/s iteration_duration.............: avg=513.3ms min=17.45ms med=451.51ms max=2.6s p(90)=960.82ms p(95)=1.18s iterations.....................: 5875 193.744005/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `apollo-router` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 16050 ✗ 0 data_received..................: 469 MB 15 MB/s data_sent......................: 6.4 MB 209 kB/s http_req_blocked...............: avg=81.76µs min=1.45µs med=2.89µs max=130.94ms p(90)=4.6µs p(95)=5.92µs http_req_connecting............: avg=32.76µs min=0s med=0s max=5.01ms p(90)=0s p(95)=0s http_req_duration..............: avg=497.51ms min=19.24ms med=492.69ms max=2.12s p(90)=621.63ms p(95)=669.47ms { expected_response:true }...: avg=497.51ms min=19.24ms med=492.69ms max=2.12s p(90)=621.63ms p(95)=669.47ms http_req_failed................: 0.00% ✓ 0 ✗ 5350 http_req_receiving.............: avg=19.33ms min=36.83µs med=69.62µs max=1.51s p(90)=34.38ms p(95)=93.67ms http_req_sending...............: avg=1.44ms min=8.04µs med=13.57µs max=352.35ms p(90)=38.47µs p(95)=552.77µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=476.73ms min=19.16ms med=483.03ms max=863.87ms p(90)=602.29ms p(95)=628.83ms http_reqs......................: 5350 175.98262/s iteration_duration.............: avg=565.33ms min=36.74ms med=541.5ms max=2.38s p(90)=725.34ms p(95)=844.85ms iterations.....................: 5350 175.98262/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `wundergraph` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 13920 ✗ 0 data_received..................: 407 MB 14 MB/s data_sent......................: 5.5 MB 182 kB/s http_req_blocked...............: avg=155.72µs min=1.19µs med=2.72µs max=331.64ms p(90)=4.21µs p(95)=5.85µs http_req_connecting............: avg=100.02µs min=0s med=0s max=331.58ms p(90)=0s p(95)=0s http_req_duration..............: avg=529.16ms min=32.39ms med=499.54ms max=2.03s p(90)=840.34ms p(95)=946.13ms { expected_response:true }...: avg=529.16ms min=32.39ms med=499.54ms max=2.03s p(90)=840.34ms p(95)=946.13ms http_req_failed................: 0.00% ✓ 0 ✗ 4640 http_req_receiving.............: avg=43.75ms min=30.3µs med=80.65µs max=1.9s p(90)=131.15ms p(95)=301.96ms http_req_sending...............: avg=2.38ms min=7.83µs med=13.24µs max=672.51ms p(90)=40.71µs p(95)=640.55µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=483.01ms min=16.31ms med=462.97ms max=1.18s p(90)=785.05ms p(95)=860.85ms http_reqs......................: 4640 153.320506/s iteration_duration.............: avg=649.5ms min=51.22ms med=598.07ms max=2.63s p(90)=1.03s p(95)=1.22s iterations.....................: 4640 153.320506/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mesh-supergraph-bun` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 7233 ✗ 0 data_received..................: 212 MB 6.9 MB/s data_sent......................: 2.9 MB 93 kB/s http_req_blocked...............: avg=55.75µs min=1.47µs med=2.93µs max=5.54ms p(90)=4.74µs p(95)=11.61µs http_req_connecting............: avg=45.64µs min=0s med=0s max=4.01ms p(90)=0s p(95)=0s http_req_duration..............: avg=1.23s min=797.17ms med=1.05s max=2.97s p(90)=1.99s p(95)=2.15s { expected_response:true }...: avg=1.23s min=797.17ms med=1.05s max=2.97s p(90)=1.99s p(95)=2.15s http_req_failed................: 0.00% ✓ 0 ✗ 2411 http_req_receiving.............: avg=11.61ms min=31.24µs med=112.97µs max=606.25ms p(90)=1.14ms p(95)=23ms http_req_sending...............: avg=350.83µs min=8.26µs med=14.03µs max=97.64ms p(90)=39.4µs p(95)=145.91µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=1.22s min=776.71ms med=1.05s max=2.97s p(90)=1.98s p(95)=2.15s http_reqs......................: 2411 78.340822/s iteration_duration.............: avg=1.26s min=806.62ms med=1.08s max=2.99s p(90)=2.02s p(95)=2.2s iterations.....................: 2411 78.340822/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `apollo-server` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 4947 ✗ 0 data_received..................: 145 MB 4.6 MB/s data_sent......................: 2.0 MB 63 kB/s http_req_blocked...............: avg=83.71µs min=1.58µs med=3.42µs max=3.8ms p(90)=5.51µs p(95)=223.61µs http_req_connecting............: avg=73.2µs min=0s med=0s max=3.77ms p(90)=0s p(95)=144.74µs http_req_duration..............: avg=1.85s min=731.55ms med=1.56s max=19.72s p(90)=1.9s p(95)=2.29s { expected_response:true }...: avg=1.85s min=731.55ms med=1.56s max=19.72s p(90)=1.9s p(95)=2.29s http_req_failed................: 0.00% ✓ 0 ✗ 1649 http_req_receiving.............: avg=186.78µs min=48.92µs med=120.61µs max=26.89ms p(90)=198.83µs p(95)=300.09µs http_req_sending...............: avg=68.61µs min=8.46µs med=18.44µs max=16.28ms p(90)=36.61µs p(95)=143.32µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=1.85s min=731.42ms med=1.56s max=19.72s p(90)=1.9s p(95)=2.29s http_reqs......................: 1649 52.727041/s iteration_duration.............: avg=1.86s min=742.17ms med=1.58s max=19.73s p(90)=1.91s p(95)=2.3s iterations.....................: 1649 52.727041/s vus............................: 38 min=38 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mercurius` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 2670 ✗ 0 data_received..................: 78 MB 2.5 MB/s data_sent......................: 1.1 MB 34 kB/s http_req_blocked...............: avg=280.7µs min=1.51µs med=4.17µs max=4.9ms p(90)=897.85µs p(95)=2.46ms http_req_connecting............: avg=266.85µs min=0s med=0s max=4.8ms p(90)=771.25µs p(95)=2.43ms http_req_duration..............: avg=3.4s min=470.01ms med=3.3s max=7.73s p(90)=4.2s p(95)=4.77s { expected_response:true }...: avg=3.4s min=470.01ms med=3.3s max=7.73s p(90)=4.2s p(95)=4.77s http_req_failed................: 0.00% ✓ 0 ✗ 890 http_req_receiving.............: avg=252.44µs min=44.65µs med=114.91µs max=44.87ms p(90)=184.99µs p(95)=239.85µs http_req_sending...............: avg=59.27µs min=9.23µs med=22.16µs max=3.59ms p(90)=84.09µs p(95)=230.82µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=3.4s min=469.72ms med=3.3s max=7.73s p(90)=4.2s p(95)=4.77s http_reqs......................: 890 28.964969/s iteration_duration.............: avg=3.42s min=485.03ms med=3.31s max=7.74s p(90)=4.22s p(95)=4.78s iterations.....................: 890 28.964969/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mesh-supergraph` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 2334 ✗ 0 data_received..................: 68 MB 2.1 MB/s data_sent......................: 924 kB 29 kB/s http_req_blocked...............: avg=271.45µs min=1.53µs med=4.09µs max=13.52ms p(90)=659.41µs p(95)=2.29ms http_req_connecting............: avg=247.67µs min=0s med=0s max=13.48ms p(90)=533.48µs p(95)=1.84ms http_req_duration..............: avg=4.03s min=1.67s med=3.83s max=8.7s p(90)=5.59s p(95)=6.41s { expected_response:true }...: avg=4.03s min=1.67s med=3.83s max=8.7s p(90)=5.59s p(95)=6.41s http_req_failed................: 0.00% ✓ 0 ✗ 778 http_req_receiving.............: avg=2.55ms min=45.04µs med=159.75µs max=75.79ms p(90)=2.74ms p(95)=5.04ms http_req_sending...............: avg=216.06µs min=8.88µs med=23.82µs max=8.94ms p(90)=131.34µs p(95)=582.11µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=4.02s min=1.66s med=3.83s max=8.7s p(90)=5.59s p(95)=6.41s http_reqs......................: 778 24.188699/s iteration_duration.............: avg=4.04s min=1.69s med=3.84s max=8.71s p(90)=5.6s p(95)=6.44s iterations.....................: 778 24.188699/s vus............................: 19 min=19 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
github-actions[bot] commented 1 day ago

Overview for: federation-v1/constant-vus-subgraphs-delay-resources

This scenario runs 4 subgraphs and a GraphQL gateway with Federation v1 spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 100 VUs over 30s

Comparison

Comparison
Gateway RPS ⬇️ Requests Duration Notes
cosmo 182 5536 total, 0 failed avg: 311ms, p95: 823ms ✅
apollo-router 166 5049 total, 0 failed avg: 333ms, p95: 870ms ✅
wundergraph 152 4629 total, 0 failed avg: 354ms, p95: 999ms ✅
mesh-supergraph-bun 98 3043 total, 0 failed avg: 974ms, p95: 1648ms ✅
apollo-server 72 2247 total, 0 failed avg: 1348ms, p95: 1554ms ✅
mesh-supergraph 55 1725 total, 0 failed avg: 1758ms, p95: 2625ms ✅
mercurius 46 1421 total, 0 failed avg: 2132ms, p95: 2637ms ✅
Summary for: `cosmo` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 16608 ✗ 0 data_received..................: 486 MB 16 MB/s data_sent......................: 6.6 MB 217 kB/s http_req_blocked...............: avg=197.95µs min=1.71µs med=3.07µs max=884.12ms p(90)=4.68µs p(95)=6.04µs http_req_connecting............: avg=190.44µs min=0s med=0s max=884.05ms p(90)=0s p(95)=0s http_req_duration..............: avg=311.39ms min=12.15ms med=243.26ms max=2.19s p(90)=668.44ms p(95)=822.69ms { expected_response:true }...: avg=311.39ms min=12.15ms med=243.26ms max=2.19s p(90)=668.44ms p(95)=822.69ms http_req_failed................: 0.00% ✓ 0 ✗ 5536 http_req_receiving.............: avg=126.42ms min=38.97µs med=96.82µs max=1.94s p(90)=479.24ms p(95)=655.92ms http_req_sending...............: avg=5.67ms min=7.64µs med=14.46µs max=1.33s p(90)=52.8µs p(95)=807.04µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=179.29ms min=9.78ms med=173.65ms max=889.74ms p(90)=313.37ms p(95)=358.56ms http_reqs......................: 5536 182.626026/s iteration_duration.............: avg=544.3ms min=23.22ms med=449.19ms max=2.92s p(90)=1.11s p(95)=1.38s iterations.....................: 5536 182.626026/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `apollo-router` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 15147 ✗ 0 data_received..................: 443 MB 15 MB/s data_sent......................: 6.0 MB 198 kB/s http_req_blocked...............: avg=224.53µs min=1.65µs med=3.42µs max=306.38ms p(90)=5.45µs p(95)=7.55µs http_req_connecting............: avg=118.64µs min=0s med=0s max=175.8ms p(90)=0s p(95)=0s http_req_duration..............: avg=333.03ms min=9.67ms med=253.05ms max=1.97s p(90)=715.39ms p(95)=870.15ms { expected_response:true }...: avg=333.03ms min=9.67ms med=253.05ms max=1.97s p(90)=715.39ms p(95)=870.15ms http_req_failed................: 0.00% ✓ 0 ✗ 5049 http_req_receiving.............: avg=120.2ms min=36.18µs med=87.89µs max=1.85s p(90)=507.68ms p(95)=675.02ms http_req_sending...............: avg=6.49ms min=8.58µs med=16.44µs max=1.85s p(90)=52.23µs p(95)=391.08µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=206.33ms min=9.47ms med=197.54ms max=676.91ms p(90)=341.85ms p(95)=402.75ms http_reqs......................: 5049 166.453115/s iteration_duration.............: avg=597.78ms min=26.29ms med=502.39ms max=3.27s p(90)=1.19s p(95)=1.47s iterations.....................: 5049 166.453115/s vus............................: 94 min=94 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `wundergraph` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 13887 ✗ 0 data_received..................: 406 MB 13 MB/s data_sent......................: 5.5 MB 182 kB/s http_req_blocked...............: avg=117µs min=1.72µs med=3.45µs max=179.52ms p(90)=5.32µs p(95)=6.78µs http_req_connecting............: avg=48.31µs min=0s med=0s max=10.53ms p(90)=0s p(95)=0s http_req_duration..............: avg=353.54ms min=11.97ms med=248.22ms max=2.74s p(90)=792.46ms p(95)=998.85ms { expected_response:true }...: avg=353.54ms min=11.97ms med=248.22ms max=2.74s p(90)=792.46ms p(95)=998.85ms http_req_failed................: 0.00% ✓ 0 ✗ 4629 http_req_receiving.............: avg=149.24ms min=42.52µs med=154.58µs max=2.71s p(90)=567.47ms p(95)=766.08ms http_req_sending...............: avg=7.22ms min=8.27µs med=16.49µs max=1.09s p(90)=126.37µs p(95)=3.68ms http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=197.07ms min=11.73ms med=185.46ms max=721.18ms p(90)=329.88ms p(95)=385.5ms http_reqs......................: 4629 152.900546/s iteration_duration.............: avg=650.93ms min=25.12ms med=483.4ms max=3.96s p(90)=1.39s p(95)=1.72s iterations.....................: 4629 152.900546/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mesh-supergraph-bun` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 9129 ✗ 0 data_received..................: 267 MB 8.6 MB/s data_sent......................: 3.6 MB 117 kB/s http_req_blocked...............: avg=44.39µs min=1.52µs med=2.91µs max=8.82ms p(90)=4.67µs p(95)=8.89µs http_req_connecting............: avg=28.08µs min=0s med=0s max=4.35ms p(90)=0s p(95)=0s http_req_duration..............: avg=973.52ms min=179.22ms med=830.12ms max=2.26s p(90)=1.55s p(95)=1.64s { expected_response:true }...: avg=973.52ms min=179.22ms med=830.12ms max=2.26s p(90)=1.55s p(95)=1.64s http_req_failed................: 0.00% ✓ 0 ✗ 3043 http_req_receiving.............: avg=9.08ms min=39.73µs med=90.45µs max=451.87ms p(90)=2.24ms p(95)=38.01ms http_req_sending...............: avg=468.1µs min=8.7µs med=13.81µs max=83.89ms p(90)=35.59µs p(95)=212.77µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=963.97ms min=178.85ms med=824.86ms max=2.13s p(90)=1.54s p(95)=1.63s http_reqs......................: 3043 98.468187/s iteration_duration.............: avg=1s min=188.87ms med=869.7ms max=2.27s p(90)=1.61s p(95)=1.69s iterations.....................: 3043 98.468187/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `apollo-server` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 6741 ✗ 0 data_received..................: 198 MB 6.4 MB/s data_sent......................: 2.7 MB 86 kB/s http_req_blocked...............: avg=71.51µs min=1.53µs med=2.96µs max=5.11ms p(90)=4.76µs p(95)=15.14µs http_req_connecting............: avg=57.91µs min=0s med=0s max=3.61ms p(90)=0s p(95)=0s http_req_duration..............: avg=1.34s min=531.98ms med=1.22s max=14s p(90)=1.41s p(95)=1.55s { expected_response:true }...: avg=1.34s min=531.98ms med=1.22s max=14s p(90)=1.41s p(95)=1.55s http_req_failed................: 0.00% ✓ 0 ✗ 2247 http_req_receiving.............: avg=156.39µs min=45.47µs med=117.18µs max=7.07ms p(90)=190.61µs p(95)=267.35µs http_req_sending...............: avg=56.94µs min=8.85µs med=14.69µs max=5.04ms p(90)=32.37µs p(95)=103.27µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=1.34s min=531.35ms med=1.22s max=14s p(90)=1.41s p(95)=1.55s http_reqs......................: 2247 72.545936/s iteration_duration.............: avg=1.36s min=544.66ms med=1.23s max=14.01s p(90)=1.42s p(95)=1.56s iterations.....................: 2247 72.545936/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mesh-supergraph` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 5175 ✗ 0 data_received..................: 151 MB 4.9 MB/s data_sent......................: 2.0 MB 66 kB/s http_req_blocked...............: avg=71.56µs min=1.44µs med=3.67µs max=7.31ms p(90)=5.55µs p(95)=301.7µs http_req_connecting............: avg=54.82µs min=0s med=0s max=3.34ms p(90)=0s p(95)=184.03µs http_req_duration..............: avg=1.75s min=680.04ms med=1.72s max=4.44s p(90)=2.36s p(95)=2.62s { expected_response:true }...: avg=1.75s min=680.04ms med=1.72s max=4.44s p(90)=2.36s p(95)=2.62s http_req_failed................: 0.00% ✓ 0 ✗ 1725 http_req_receiving.............: avg=1.42ms min=39.65µs med=146.24µs max=219.7ms p(90)=2.89ms p(95)=5.13ms http_req_sending...............: avg=178.97µs min=7.85µs med=19.86µs max=26.51ms p(90)=51.51µs p(95)=249.13µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=1.75s min=679.92ms med=1.72s max=4.44s p(90)=2.36s p(95)=2.62s http_reqs......................: 1725 55.285634/s iteration_duration.............: avg=1.78s min=712.12ms med=1.74s max=4.47s p(90)=2.39s p(95)=2.65s iterations.....................: 1725 55.285634/s vus............................: 36 min=36 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mercurius` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 4263 ✗ 0 data_received..................: 125 MB 4.1 MB/s data_sent......................: 1.7 MB 55 kB/s http_req_blocked...............: avg=86.69µs min=1.92µs med=3.99µs max=5.42ms p(90)=5.69µs p(95)=367.86µs http_req_connecting............: avg=67.13µs min=0s med=0s max=5.05ms p(90)=0s p(95)=249.47µs http_req_duration..............: avg=2.13s min=358.94ms med=2.09s max=4.67s p(90)=2.52s p(95)=2.63s { expected_response:true }...: avg=2.13s min=358.94ms med=2.09s max=4.67s p(90)=2.52s p(95)=2.63s http_req_failed................: 0.00% ✓ 0 ✗ 1421 http_req_receiving.............: avg=214.3µs min=45.81µs med=112.74µs max=27.32ms p(90)=232.19µs p(95)=365.46µs http_req_sending...............: avg=66.7µs min=9.4µs med=21.68µs max=5.83ms p(90)=39.97µs p(95)=211.91µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=2.13s min=358.57ms med=2.09s max=4.67s p(90)=2.52s p(95)=2.63s http_reqs......................: 1421 46.212786/s iteration_duration.............: avg=2.14s min=369.82ms med=2.1s max=4.7s p(90)=2.54s p(95)=2.64s iterations.....................: 1421 46.212786/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
github-actions[bot] commented 1 day ago

Overview for: federation-v1/constant-vus-subgraphs-delay

This scenario runs 4 subgraphs and a GraphQL gateway with Federation v1 spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 100 VUs over 30s

Comparison

Comparison
Gateway RPS ⬇️ Requests Duration Notes
cosmo 196 5948 total, 0 failed avg: 337ms, p95: 821ms ✅
apollo-router 174 5334 total, 0 failed avg: 468ms, p95: 684ms ✅
wundergraph 153 4678 total, 0 failed avg: 547ms, p95: 988ms ✅
mesh-supergraph-bun 78 2426 total, 0 failed avg: 1228ms, p95: 2107ms ✅
apollo-server 54 1714 total, 0 failed avg: 1780ms, p95: 1924ms ✅
mercurius 27 851 total, 0 failed avg: 3584ms, p95: 5064ms ✅
mesh-supergraph 24 792 total, 0 failed avg: 3888ms, p95: 6016ms ✅
Summary for: `cosmo` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 17844 ✗ 0 data_received..................: 522 MB 17 MB/s data_sent......................: 7.1 MB 233 kB/s http_req_blocked...............: avg=137.54µs min=1.6µs med=2.73µs max=507.63ms p(90)=4.14µs p(95)=5.34µs http_req_connecting............: avg=22.82µs min=0s med=0s max=4.29ms p(90)=0s p(95)=0s http_req_duration..............: avg=336.58ms min=10.5ms med=276.87ms max=1.67s p(90)=670.69ms p(95)=821.31ms { expected_response:true }...: avg=336.58ms min=10.5ms med=276.87ms max=1.67s p(90)=670.69ms p(95)=821.31ms http_req_failed................: 0.00% ✓ 0 ✗ 5948 http_req_receiving.............: avg=96.68ms min=35.33µs med=82.8µs max=1.48s p(90)=422.77ms p(95)=623.5ms http_req_sending...............: avg=5.7ms min=8.54µs med=13.19µs max=1.56s p(90)=55.04µs p(95)=1.22ms http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=234.19ms min=6.06ms med=216.27ms max=781.58ms p(90)=406.53ms p(95)=493.12ms http_reqs......................: 5948 196.278853/s iteration_duration.............: avg=507.1ms min=22.16ms med=428.7ms max=2.94s p(90)=979.64ms p(95)=1.19s iterations.....................: 5948 196.278853/s vus............................: 94 min=94 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `apollo-router` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 16002 ✗ 0 data_received..................: 468 MB 15 MB/s data_sent......................: 6.3 MB 207 kB/s http_req_blocked...............: avg=23.28µs min=1.5µs med=2.96µs max=3.36ms p(90)=4.6µs p(95)=5.76µs http_req_connecting............: avg=17.52µs min=0s med=0s max=3.33ms p(90)=0s p(95)=0s http_req_duration..............: avg=468.49ms min=23.07ms med=468.68ms max=1.6s p(90)=611.37ms p(95)=684.08ms { expected_response:true }...: avg=468.49ms min=23.07ms med=468.68ms max=1.6s p(90)=611.37ms p(95)=684.08ms http_req_failed................: 0.00% ✓ 0 ✗ 5334 http_req_receiving.............: avg=27.52ms min=36.57µs med=72.59µs max=1.04s p(90)=60.2ms p(95)=162.26ms http_req_sending...............: avg=1.67ms min=7.85µs med=14µs max=503.94ms p(90)=41.73µs p(95)=572.91µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=439.3ms min=22.98ms med=454.97ms max=815.78ms p(90)=572ms p(95)=608.82ms http_reqs......................: 5334 174.500978/s iteration_duration.............: avg=568.48ms min=38.49ms med=524.43ms max=2.56s p(90)=795.73ms p(95)=1.03s iterations.....................: 5334 174.500978/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `wundergraph` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 14034 ✗ 0 data_received..................: 411 MB 14 MB/s data_sent......................: 5.6 MB 182 kB/s http_req_blocked...............: avg=45.65µs min=1.24µs med=2.95µs max=36.35ms p(90)=4.7µs p(95)=6.34µs http_req_connecting............: avg=32.74µs min=0s med=0s max=7.38ms p(90)=0s p(95)=0s http_req_duration..............: avg=547.1ms min=28.85ms med=525.91ms max=1.88s p(90)=849.2ms p(95)=987.8ms { expected_response:true }...: avg=547.1ms min=28.85ms med=525.91ms max=1.88s p(90)=849.2ms p(95)=987.8ms http_req_failed................: 0.00% ✓ 0 ✗ 4678 http_req_receiving.............: avg=34.66ms min=37.84µs med=81.56µs max=1.07s p(90)=100.33ms p(95)=250.5ms http_req_sending...............: avg=2.65ms min=7.44µs med=13.51µs max=451.39ms p(90)=40.16µs p(95)=1.15ms http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=509.78ms min=22.02ms med=488.79ms max=1.35s p(90)=808.97ms p(95)=943.9ms http_reqs......................: 4678 153.689255/s iteration_duration.............: avg=647.35ms min=42.16ms med=593.13ms max=2.5s p(90)=1.02s p(95)=1.23s iterations.....................: 4678 153.689255/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mesh-supergraph-bun` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 7278 ✗ 0 data_received..................: 213 MB 6.9 MB/s data_sent......................: 2.9 MB 94 kB/s http_req_blocked...............: avg=36.53µs min=1.42µs med=2.89µs max=3.23ms p(90)=4.73µs p(95)=13.38µs http_req_connecting............: avg=28.48µs min=0s med=0s max=2.47ms p(90)=0s p(95)=0s http_req_duration..............: avg=1.22s min=399.49ms med=1.05s max=3.34s p(90)=1.98s p(95)=2.1s { expected_response:true }...: avg=1.22s min=399.49ms med=1.05s max=3.34s p(90)=1.98s p(95)=2.1s http_req_failed................: 0.00% ✓ 0 ✗ 2426 http_req_receiving.............: avg=11.16ms min=37.74µs med=108.09µs max=619.49ms p(90)=959.21µs p(95)=48.2ms http_req_sending...............: avg=286.6µs min=7.94µs med=13.96µs max=57.72ms p(90)=36.41µs p(95)=135.38µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=1.21s min=399.13ms med=1.05s max=3.34s p(90)=1.97s p(95)=2.1s http_reqs......................: 2426 78.959391/s iteration_duration.............: avg=1.25s min=408.9ms med=1.08s max=3.37s p(90)=2s p(95)=2.15s iterations.....................: 2426 78.959391/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `apollo-server` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 5142 ✗ 0 data_received..................: 151 MB 4.8 MB/s data_sent......................: 2.0 MB 65 kB/s http_req_blocked...............: avg=94.36µs min=1.52µs med=3.03µs max=4.88ms p(90)=4.99µs p(95)=142.2µs http_req_connecting............: avg=85.32µs min=0s med=0s max=4.85ms p(90)=0s p(95)=103.1µs http_req_duration..............: avg=1.78s min=800.17ms med=1.56s max=18.45s p(90)=1.72s p(95)=1.92s { expected_response:true }...: avg=1.78s min=800.17ms med=1.56s max=18.45s p(90)=1.72s p(95)=1.92s http_req_failed................: 0.00% ✓ 0 ✗ 1714 http_req_receiving.............: avg=171.77µs min=46.83µs med=117.28µs max=19.51ms p(90)=189.56µs p(95)=265.8µs http_req_sending...............: avg=36.43µs min=8.62µs med=15.31µs max=2.91ms p(90)=32.04µs p(95)=76.6µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=1.77s min=799.96ms med=1.56s max=18.45s p(90)=1.72s p(95)=1.92s http_reqs......................: 1714 54.94944/s iteration_duration.............: avg=1.79s min=807.01ms med=1.57s max=18.46s p(90)=1.73s p(95)=1.93s iterations.....................: 1714 54.94944/s vus............................: 30 min=30 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mercurius` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 2553 ✗ 0 data_received..................: 75 MB 2.4 MB/s data_sent......................: 1.0 MB 33 kB/s http_req_blocked...............: avg=122.88µs min=1.93µs med=4.66µs max=4.93ms p(90)=179µs p(95)=949.28µs http_req_connecting............: avg=101.76µs min=0s med=0s max=3ms p(90)=147.61µs p(95)=911.59µs http_req_duration..............: avg=3.58s min=896.54ms med=3.41s max=7.98s p(90)=4.56s p(95)=5.06s { expected_response:true }...: avg=3.58s min=896.54ms med=3.41s max=7.98s p(90)=4.56s p(95)=5.06s http_req_failed................: 0.00% ✓ 0 ✗ 851 http_req_receiving.............: avg=257.03µs min=45.3µs med=122.2µs max=25.26ms p(90)=228.93µs p(95)=429.02µs http_req_sending...............: avg=68.41µs min=9.96µs med=25.53µs max=2.58ms p(90)=59.63µs p(95)=473.64µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=3.58s min=894.86ms med=3.41s max=7.98s p(90)=4.56s p(95)=5.06s http_reqs......................: 851 27.582839/s iteration_duration.............: avg=3.59s min=904.6ms med=3.42s max=7.99s p(90)=4.57s p(95)=5.07s iterations.....................: 851 27.582839/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mesh-supergraph` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 2376 ✗ 0 data_received..................: 70 MB 2.2 MB/s data_sent......................: 940 kB 30 kB/s http_req_blocked...............: avg=122.67µs min=1.34µs med=3.56µs max=7.95ms p(90)=175.4µs p(95)=910.72µs http_req_connecting............: avg=101.2µs min=0s med=0s max=7.92ms p(90)=132.2µs p(95)=784.79µs http_req_duration..............: avg=3.88s min=1.19s med=3.71s max=8.26s p(90)=5.65s p(95)=6.01s { expected_response:true }...: avg=3.88s min=1.19s med=3.71s max=8.26s p(90)=5.65s p(95)=6.01s http_req_failed................: 0.00% ✓ 0 ✗ 792 http_req_receiving.............: avg=1.42ms min=43.64µs med=125.92µs max=77.47ms p(90)=2.02ms p(95)=3.33ms http_req_sending...............: avg=244.04µs min=8.06µs med=20.4µs max=15.04ms p(90)=107.65µs p(95)=292.95µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=3.88s min=1.17s med=3.71s max=8.25s p(90)=5.65s p(95)=6.01s http_reqs......................: 792 24.852771/s iteration_duration.............: avg=3.9s min=1.2s med=3.73s max=8.27s p(90)=5.67s p(95)=6.03s iterations.....................: 792 24.852771/s vus............................: 47 min=47 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
github-actions[bot] commented 1 day ago

Overview for: federation-v1/ramping-vus

This scenario runs 4 subgraphs and a GraphQL gateway with Federation v1 spec, and runs a heavy query. We are running a heavy load of concurrent VUs to measure response time and other stats, during stress. It measure things like memory usage, CPU usage, response times. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was trying to reach 500 concurrent VUs over 60s

Comparison

Comparison
Gateway duration(p95)⬇️ RPS Requests Durations Notes
cosmo 2031ms 179 12532 total, 0 failed avg: 715ms, p95: 2031ms, max: 5874ms, med: 511ms ✅
apollo-router 2068ms 168 11809 total, 0 failed avg: 786ms, p95: 2069ms, max: 5592ms, med: 602ms ✅
wundergraph 2426ms 157 11057 total, 0 failed avg: 843ms, p95: 2426ms, max: 6148ms, med: 620ms ✅
mesh-supergraph-bun 6158ms 99 6981 total, 0 failed avg: 2660ms, p95: 6158ms, max: 11263ms, med: 2461ms ✅
mesh-supergraph 8056ms 58 4111 total, 0 failed avg: 4679ms, p95: 8056ms, max: 10141ms, med: 4865ms ✅
apollo-server 21782ms 72 5267 total, 0 failed avg: 3941ms, p95: 21783ms, max: 44039ms, med: 2156ms ✅
Summary for: `cosmo` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 37596 ✗ 0 data_received..................: 1.1 GB 16 MB/s data_sent......................: 15 MB 213 kB/s http_req_blocked...............: avg=19.47ms min=1.64µs med=3.77µs max=4.89s p(90)=5.94µs p(95)=16.54µs http_req_connecting............: avg=19.06ms min=0s med=0s max=4.89s p(90)=0s p(95)=0s http_req_duration..............: avg=715.28ms min=4.42ms med=510.57ms max=5.87s p(90)=1.62s p(95)=2.03s { expected_response:true }...: avg=715.28ms min=4.42ms med=510.57ms max=5.87s p(90)=1.62s p(95)=2.03s http_req_failed................: 0.00% ✓ 0 ✗ 12532 http_req_receiving.............: avg=222.51ms min=34.6µs med=94.69µs max=5.02s p(90)=987.01ms p(95)=1.26s http_req_sending...............: avg=34.36ms min=8.3µs med=19.94µs max=4.12s p(90)=10.87ms p(95)=139.27ms http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=458.39ms min=4.32ms med=361.13ms max=3.31s p(90)=970.56ms p(95)=1.25s http_reqs......................: 12532 179.01019/s iteration_duration.............: avg=1.45s min=10.15ms med=1.03s max=8.55s p(90)=3.36s p(95)=4.28s iterations.....................: 12532 179.01019/s vus............................: 1 min=1 max=495 vus_max........................: 500 min=500 max=500 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `apollo-router` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 35427 ✗ 0 data_received..................: 1.0 GB 15 MB/s data_sent......................: 14 MB 200 kB/s http_req_blocked...............: avg=19.52ms min=1.53µs med=3.3µs max=3.31s p(90)=5.3µs p(95)=21.76µs http_req_connecting............: avg=18.72ms min=0s med=0s max=3.31s p(90)=0s p(95)=0s http_req_duration..............: avg=786.14ms min=6.98ms med=601.74ms max=5.59s p(90)=1.71s p(95)=2.06s { expected_response:true }...: avg=786.14ms min=6.98ms med=601.74ms max=5.59s p(90)=1.71s p(95)=2.06s http_req_failed................: 0.00% ✓ 0 ✗ 11809 http_req_receiving.............: avg=244.28ms min=32.85µs med=82.21µs max=4.09s p(90)=1.04s p(95)=1.34s http_req_sending...............: avg=32.59ms min=8.64µs med=15.76µs max=4.25s p(90)=771.74µs p(95)=86.34ms http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=509.26ms min=6.88ms med=416.28ms max=2.8s p(90)=1.02s p(95)=1.21s http_reqs......................: 11809 168.66422/s iteration_duration.............: avg=1.54s min=13.15ms med=1.14s max=10.09s p(90)=3.53s p(95)=4.4s iterations.....................: 11809 168.66422/s vus............................: 1 min=1 max=500 vus_max........................: 500 min=500 max=500 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `wundergraph` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 33171 ✗ 0 data_received..................: 970 MB 14 MB/s data_sent......................: 13 MB 188 kB/s http_req_blocked...............: avg=24.7ms min=1.62µs med=3.3µs max=4.48s p(90)=5.58µs p(95)=55.31µs http_req_connecting............: avg=23.97ms min=0s med=0s max=4.48s p(90)=0s p(95)=0s http_req_duration..............: avg=843.42ms min=6.55ms med=620.38ms max=6.14s p(90)=1.82s p(95)=2.42s { expected_response:true }...: avg=843.42ms min=6.55ms med=620.38ms max=6.14s p(90)=1.82s p(95)=2.42s http_req_failed................: 0.00% ✓ 0 ✗ 11057 http_req_receiving.............: avg=290ms min=34.39µs med=95.4µs max=4.03s p(90)=1.12s p(95)=1.57s http_req_sending...............: avg=35.46ms min=8.43µs med=15.59µs max=3.94s p(90)=530.71µs p(95)=86.12ms http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=517.95ms min=6.44ms med=383.69ms max=3.45s p(90)=1.1s p(95)=1.35s http_reqs......................: 11057 157.953641/s iteration_duration.............: avg=1.66s min=12.59ms med=1.24s max=11.35s p(90)=3.78s p(95)=4.85s iterations.....................: 11057 157.953641/s vus............................: 1 min=1 max=499 vus_max........................: 500 min=500 max=500 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mesh-supergraph-bun` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 20943 ✗ 0 data_received..................: 612 MB 8.7 MB/s data_sent......................: 8.3 MB 118 kB/s http_req_blocked...............: avg=2.52ms min=1.47µs med=3.02µs max=575.07ms p(90)=5.85µs p(95)=253.87µs http_req_connecting............: avg=2.44ms min=0s med=0s max=504.94ms p(90)=0s p(95)=169.18µs http_req_duration..............: avg=2.65s min=5.93ms med=2.46s max=11.26s p(90)=4.96s p(95)=6.15s { expected_response:true }...: avg=2.65s min=5.93ms med=2.46s max=11.26s p(90)=4.96s p(95)=6.15s http_req_failed................: 0.00% ✓ 0 ✗ 6981 http_req_receiving.............: avg=27.6ms min=36.49µs med=75.19µs max=1.52s p(90)=25.51ms p(95)=191.23ms http_req_sending...............: avg=2.54ms min=8.01µs med=14.31µs max=548.1ms p(90)=86.15µs p(95)=9.84ms http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=2.62s min=5.77ms med=2.44s max=11.05s p(90)=4.87s p(95)=6.14s http_reqs......................: 6981 99.727231/s iteration_duration.............: avg=2.73s min=11.62ms med=2.51s max=11.45s p(90)=4.98s p(95)=6.22s iterations.....................: 6981 99.727231/s vus............................: 1 min=1 max=500 vus_max........................: 500 min=500 max=500 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mesh-supergraph` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 12333 ✗ 0 data_received..................: 361 MB 5.2 MB/s data_sent......................: 4.9 MB 70 kB/s http_req_blocked...............: avg=335.66µs min=1.37µs med=3.71µs max=110.74ms p(90)=187.42µs p(95)=376.17µs http_req_connecting............: avg=315.33µs min=0s med=0s max=110.68ms p(90)=129.21µs p(95)=294.19µs http_req_duration..............: avg=4.67s min=12.92ms med=4.86s max=10.14s p(90)=7.54s p(95)=8.05s { expected_response:true }...: avg=4.67s min=12.92ms med=4.86s max=10.14s p(90)=7.54s p(95)=8.05s http_req_failed................: 0.00% ✓ 0 ✗ 4111 http_req_receiving.............: avg=1.2ms min=36.88µs med=118.22µs max=120.22ms p(90)=2.58ms p(95)=4.58ms http_req_sending...............: avg=277.02µs min=8.22µs med=20.62µs max=155.85ms p(90)=48.99µs p(95)=75.28µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=4.67s min=12.83ms med=4.86s max=10.14s p(90)=7.54s p(95)=8.05s http_reqs......................: 4111 58.72261/s iteration_duration.............: avg=4.71s min=18.54ms med=4.89s max=10.16s p(90)=7.57s p(95)=8.07s iterations.....................: 4111 58.72261/s vus............................: 1 min=1 max=499 vus_max........................: 500 min=500 max=500 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `apollo-server` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 15801 ✗ 0 data_received..................: 463 MB 6.4 MB/s data_sent......................: 6.3 MB 86 kB/s http_req_blocked...............: avg=133.52µs min=1.49µs med=3.06µs max=31.63ms p(90)=20.28µs p(95)=243.92µs http_req_connecting............: avg=118.29µs min=0s med=0s max=31.54ms p(90)=0s p(95)=165.15µs http_req_duration..............: avg=3.94s min=97.95ms med=2.15s max=44.03s p(90)=3.38s p(95)=21.78s { expected_response:true }...: avg=3.94s min=97.95ms med=2.15s max=44.03s p(90)=3.38s p(95)=21.78s http_req_failed................: 0.00% ✓ 0 ✗ 5267 http_req_receiving.............: avg=335.84µs min=46.75µs med=115.02µs max=125.76ms p(90)=202.07µs p(95)=307.47µs http_req_sending...............: avg=72.68µs min=8.67µs med=15.59µs max=33.59ms p(90)=39.8µs p(95)=57.76µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=3.94s min=97.86ms med=2.15s max=44.03s p(90)=3.38s p(95)=21.78s http_reqs......................: 5267 72.329188/s iteration_duration.............: avg=3.95s min=106.07ms med=2.16s max=44.06s p(90)=3.39s p(95)=21.79s iterations.....................: 5267 72.329188/s vus............................: 64 min=57 max=500 vus_max........................: 500 min=500 max=500 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview