search

the-guild-org / gateways-benchmark

MIT License
43 stars 7 forks source link

Update dependency typescript to v5.6.2 #435

Closed renovate[bot] closed 3 days ago

renovate[bot] commented 6 months ago

This PR contains the following updates:

Package Change Age Adoption Passing Confidence
typescript (source) 5.4.3 -> 5.6.2 age adoption passing confidence

Release Notes

microsoft/TypeScript (typescript) ### [`v5.6.2`](https://redirect.github.com/microsoft/TypeScript/compare/v5.5.4...a7e3374f13327483fbe94e32806d65785b0b6cda) [Compare Source](https://redirect.github.com/microsoft/TypeScript/compare/v5.5.4...v5.6.2) ### [`v5.5.4`](https://redirect.github.com/microsoft/TypeScript/releases/tag/v5.5.4): TypeScript 5.5.4 [Compare Source](https://redirect.github.com/microsoft/TypeScript/compare/v5.5.3...v5.5.4) For release notes, check out the [release announcement](https://devblogs.microsoft.com/typescript/announcing-typescript-5-5/). For the complete list of fixed issues, check out the - [fixed issues query for TypeScript v5.5.4 (Stable)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=is%3Aissue+milestone%3A%22TypeScript+5.5.4%22+is%3Aclosed+). - [fixed issues query for TypeScript v5.5.3 (Stable)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=is%3Aissue+milestone%3A%22TypeScript+5.5.3%22+is%3Aclosed+). - [fixed issues query for TypeScript v5.5.2 (Stable)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=is%3Aissue+milestone%3A%22TypeScript+5.5.2%22+is%3Aclosed+). - [fixed issues query for TypeScript v5.5.1 (RC)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=is%3Aissue+milestone%3A%22TypeScript+5.5.1%22+is%3Aclosed+). - [fixed issues query for TypeScript v5.5.0 (Beta)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=is%3Aissue+milestone%3A%22TypeScript+5.5.0%22+is%3Aclosed+). Downloads are available on: - [npm](https://www.npmjs.com/package/typescript) - [NuGet package](https://www.nuget.org/packages/Microsoft.TypeScript.MSBuild) (soon!) ### [`v5.5.3`](https://redirect.github.com/microsoft/TypeScript/releases/tag/v5.5.3): TypeScript 5.5.3 [Compare Source](https://redirect.github.com/microsoft/TypeScript/compare/v5.5.2...v5.5.3) For release notes, check out the [release announcement](https://devblogs.microsoft.com/typescript/announcing-typescript-5-5/). For the complete list of fixed issues, check out the - [fixed issues query for TypeScript v5.5.3 (Stable)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=is%3Aissue+milestone%3A%22TypeScript+5.5.3%22+is%3Aclosed+). - [fixed issues query for TypeScript v5.5.2 (Stable)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=is%3Aissue+milestone%3A%22TypeScript+5.5.2%22+is%3Aclosed+). - [fixed issues query for TypeScript v5.5.1 (RC)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=is%3Aissue+milestone%3A%22TypeScript+5.5.1%22+is%3Aclosed+). - [fixed issues query for TypeScript v5.5.0 (Beta)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=is%3Aissue+milestone%3A%22TypeScript+5.5.0%22+is%3Aclosed+). Downloads are available on: - [npm](https://www.npmjs.com/package/typescript) - [NuGet package](https://www.nuget.org/packages/Microsoft.TypeScript.MSBuild) ### [`v5.5.2`](https://redirect.github.com/microsoft/TypeScript/releases/tag/v5.5.2): TypeScript 5.5 [Compare Source](https://redirect.github.com/microsoft/TypeScript/compare/v5.4.5...v5.5.2) For release notes, check out the [release announcement](https://devblogs.microsoft.com/typescript/announcing-typescript-5-5/). For the complete list of fixed issues, check out the - [fixed issues query for TypeScript v5.5.2 (Stable)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=is%3Aissue+milestone%3A%22TypeScript+5.5.2%22+is%3Aclosed+). - [fixed issues query for TypeScript v5.5.1 (RC)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=is%3Aissue+milestone%3A%22TypeScript+5.5.1%22+is%3Aclosed+). - [fixed issues query for TypeScript v5.5.0 (Beta)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=is%3Aissue+milestone%3A%22TypeScript+5.5.0%22+is%3Aclosed+). Downloads are available on: - [npm](https://www.npmjs.com/package/typescript) ### [`v5.4.5`](https://redirect.github.com/microsoft/TypeScript/releases/tag/v5.4.5): TypeScript 5.4.5 [Compare Source](https://redirect.github.com/microsoft/TypeScript/compare/v5.4.4...v5.4.5) For release notes, check out the [release announcement](https://devblogs.microsoft.com/typescript/announcing-typescript-5-4/). For the complete list of fixed issues, check out the - [fixed issues query for Typescript 5.4.0 (Beta)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=milestone%3A%22TypeScript+5.4.0%22+is%3Aclosed+). - [fixed issues query for Typescript 5.4.1 (RC)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=milestone%3A%22TypeScript+5.4.1%22+is%3Aclosed+). - [fixed issues query for Typescript 5.4.2 (Stable)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=milestone%3A%22TypeScript+5.4.2%22+is%3Aclosed+). - [fixed issues query for Typescript 5.4.3 (Stable)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=milestone%3A%22TypeScript+5.4.3%22+is%3Aclosed+). - [fixed issues query for Typescript 5.4.4 (Stable)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=milestone%3A%22TypeScript+5.4.4%22+is%3Aclosed+). - [fixed issues query for Typescript 5.4.5 (Stable)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=milestone%3A%22TypeScript+5.4.5%22+is%3Aclosed+). Downloads are available on: - [NuGet package](https://www.nuget.org/packages/Microsoft.TypeScript.MSBuild) ### [`v5.4.4`](https://redirect.github.com/microsoft/TypeScript/releases/tag/v5.4.4): TypeScript 5.4.4 [Compare Source](https://redirect.github.com/microsoft/TypeScript/compare/v5.4.3...v5.4.4) For release notes, check out the [release announcement](https://devblogs.microsoft.com/typescript/announcing-typescript-5-4/). For the complete list of fixed issues, check out the - [fixed issues query for Typescript 5.4.0 (Beta)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=milestone%3A%22TypeScript+5.4.0%22+is%3Aclosed+). - [fixed issues query for Typescript 5.4.1 (RC)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=milestone%3A%22TypeScript+5.4.1%22+is%3Aclosed+). - [fixed issues query for Typescript 5.4.2 (Stable)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=milestone%3A%22TypeScript+5.4.2%22+is%3Aclosed+). - [fixed issues query for Typescript 5.4.3 (Stable)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=milestone%3A%22TypeScript+5.4.3%22+is%3Aclosed+). - [fixed issues query for Typescript 5.4.4 (Stable)](https://redirect.github.com/Microsoft/TypeScript/issues?utf8=%E2%9C%93\&q=milestone%3A%22TypeScript+5.4.4%22+is%3Aclosed+). Downloads are available on: - [NuGet package](https://www.nuget.org/packages/Microsoft.TypeScript.MSBuild)

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about these updates again.

This PR was generated by Mend Renovate. View the repository job log.

github-actions[bot] commented 4 days ago

Overview for: federation-v1/constant-vus-subgraphs-delay-resources

This scenario runs 4 subgraphs and a GraphQL gateway with Federation v1 spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 100 VUs over 30s

Comparison

Comparison
Gateway RPS ⬇️ Requests Duration Notes
cosmo 183 5559 total, 0 failed avg: 296ms, p95: 898ms ✅
apollo-router 162 4930 total, 0 failed avg: 344ms, p95: 923ms ✅
wundergraph 155 4720 total, 0 failed avg: 342ms, p95: 922ms ✅
mesh-supergraph-bun 86 2661 total, 0 failed avg: 1105ms, p95: 1810ms ✅
apollo-server 72 2253 total, 0 failed avg: 1345ms, p95: 1565ms ✅
mesh-supergraph 54 1709 total, 0 failed avg: 1771ms, p95: 2555ms ✅
mercurius 47 1447 total, 0 failed avg: 2088ms, p95: 2723ms ✅
Summary for: `cosmo` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 16677 ✗ 0 data_received..................: 488 MB 16 MB/s data_sent......................: 6.6 MB 218 kB/s http_req_blocked...............: avg=371.78µs min=1.4µs med=3.04µs max=1.07s p(90)=4.34µs p(95)=5.43µs http_req_connecting............: avg=331.15µs min=0s med=0s max=1.07s p(90)=0s p(95)=0s http_req_duration..............: avg=296.22ms min=7.94ms med=196.08ms max=2.08s p(90)=688.8ms p(95)=898.28ms { expected_response:true }...: avg=296.22ms min=7.94ms med=196.08ms max=2.08s p(90)=688.8ms p(95)=898.28ms http_req_failed................: 0.00% ✓ 0 ✗ 5559 http_req_receiving.............: avg=138.09ms min=39.32µs med=97.95µs max=2.01s p(90)=525.63ms p(95)=706.23ms http_req_sending...............: avg=5.57ms min=8.46µs med=13.75µs max=1.07s p(90)=52.05µs p(95)=1.81ms http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=152.55ms min=7.68ms med=139.13ms max=831.27ms p(90)=278.04ms p(95)=323.01ms http_reqs......................: 5559 183.19909/s iteration_duration.............: avg=542.9ms min=19.14ms med=386.25ms max=3.98s p(90)=1.21s p(95)=1.48s iterations.....................: 5559 183.19909/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `apollo-router` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 14790 ✗ 0 data_received..................: 433 MB 14 MB/s data_sent......................: 5.9 MB 193 kB/s http_req_blocked...............: avg=284.32µs min=1.62µs med=3.45µs max=370.76ms p(90)=5.49µs p(95)=6.38µs http_req_connecting............: avg=200.61µs min=0s med=0s max=369.26ms p(90)=0s p(95)=0s http_req_duration..............: avg=344.19ms min=15.53ms med=262.04ms max=2.32s p(90)=745.54ms p(95)=922.68ms { expected_response:true }...: avg=344.19ms min=15.53ms med=262.04ms max=2.32s p(90)=745.54ms p(95)=922.68ms http_req_failed................: 0.00% ✓ 0 ✗ 4930 http_req_receiving.............: avg=134.2ms min=35.54µs med=112.5µs max=2.09s p(90)=518.28ms p(95)=686.66ms http_req_sending...............: avg=4.89ms min=9µs med=17.05µs max=1.16s p(90)=85.99µs p(95)=549.9µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=205.09ms min=14.07ms med=197.81ms max=656.83ms p(90)=350.8ms p(95)=408.2ms http_reqs......................: 4930 162.678141/s iteration_duration.............: avg=611.72ms min=26.18ms med=504.12ms max=4.07s p(90)=1.23s p(95)=1.51s iterations.....................: 4930 162.678141/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `wundergraph` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 14160 ✗ 0 data_received..................: 414 MB 14 MB/s data_sent......................: 5.6 MB 185 kB/s http_req_blocked...............: avg=210.35µs min=1.74µs med=3.06µs max=299.67ms p(90)=4.28µs p(95)=5.91µs http_req_connecting............: avg=102.78µs min=0s med=0s max=299.61ms p(90)=0s p(95)=0s http_req_duration..............: avg=342.2ms min=13.45ms med=263.89ms max=2.61s p(90)=709.24ms p(95)=921.78ms { expected_response:true }...: avg=342.2ms min=13.45ms med=263.89ms max=2.61s p(90)=709.24ms p(95)=921.78ms http_req_failed................: 0.00% ✓ 0 ✗ 4720 http_req_receiving.............: avg=125.12ms min=38.89µs med=86.24µs max=2.02s p(90)=509.61ms p(95)=707.2ms http_req_sending...............: avg=5.83ms min=8.37µs med=13.89µs max=1.08s p(90)=70.36µs p(95)=528.33µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=211.24ms min=10.2ms med=204.79ms max=772.56ms p(90)=354.96ms p(95)=396.28ms http_reqs......................: 4720 155.929802/s iteration_duration.............: avg=638.27ms min=26.56ms med=518.7ms max=3.05s p(90)=1.3s p(95)=1.58s iterations.....................: 4720 155.929802/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mesh-supergraph-bun` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 7983 ✗ 0 data_received..................: 233 MB 7.6 MB/s data_sent......................: 3.2 MB 102 kB/s http_req_blocked...............: avg=79.73µs min=1.52µs med=2.88µs max=4.73ms p(90)=4.75µs p(95)=10.16µs http_req_connecting............: avg=73.43µs min=0s med=0s max=4.63ms p(90)=0s p(95)=0s http_req_duration..............: avg=1.1s min=310.6ms med=928.75ms max=2.45s p(90)=1.72s p(95)=1.8s { expected_response:true }...: avg=1.1s min=310.6ms med=928.75ms max=2.45s p(90)=1.72s p(95)=1.8s http_req_failed................: 0.00% ✓ 0 ✗ 2661 http_req_receiving.............: avg=12.61ms min=36.06µs med=95µs max=577.52ms p(90)=9.96ms p(95)=82.29ms http_req_sending...............: avg=665.45µs min=8.78µs med=13.72µs max=159.24ms p(90)=39.59µs p(95)=256.82µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=1.09s min=310.24ms med=917.99ms max=2.45s p(90)=1.7s p(95)=1.79s http_reqs......................: 2661 86.235347/s iteration_duration.............: avg=1.14s min=322.49ms med=974.41ms max=2.47s p(90)=1.77s p(95)=1.86s iterations.....................: 2661 86.235347/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `apollo-server` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 6759 ✗ 0 data_received..................: 198 MB 6.4 MB/s data_sent......................: 2.7 MB 86 kB/s http_req_blocked...............: avg=150.52µs min=1.39µs med=2.94µs max=6.26ms p(90)=4.61µs p(95)=14.22µs http_req_connecting............: avg=144.87µs min=0s med=0s max=6.18ms p(90)=0s p(95)=0s http_req_duration..............: avg=1.34s min=536.38ms med=1.23s max=14.12s p(90)=1.43s p(95)=1.56s { expected_response:true }...: avg=1.34s min=536.38ms med=1.23s max=14.12s p(90)=1.43s p(95)=1.56s http_req_failed................: 0.00% ✓ 0 ✗ 2253 http_req_receiving.............: avg=171.2µs min=46.57µs med=112.8µs max=5.96ms p(90)=197.48µs p(95)=278.89µs http_req_sending...............: avg=38.52µs min=8.75µs med=15.15µs max=5.37ms p(90)=33.05µs p(95)=83.67µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=1.34s min=536.15ms med=1.23s max=14.12s p(90)=1.43s p(95)=1.56s http_reqs......................: 2253 72.60359/s iteration_duration.............: avg=1.35s min=545.46ms med=1.24s max=14.14s p(90)=1.44s p(95)=1.58s iterations.....................: 2253 72.60359/s vus............................: 5 min=5 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mesh-supergraph` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 5127 ✗ 0 data_received..................: 150 MB 4.8 MB/s data_sent......................: 2.0 MB 65 kB/s http_req_blocked...............: avg=125.01µs min=1.46µs med=3.67µs max=9.02ms p(90)=5.57µs p(95)=302.09µs http_req_connecting............: avg=105.47µs min=0s med=0s max=5.06ms p(90)=0s p(95)=129.26µs http_req_duration..............: avg=1.77s min=642.84ms med=1.71s max=3.44s p(90)=2.3s p(95)=2.55s { expected_response:true }...: avg=1.77s min=642.84ms med=1.71s max=3.44s p(90)=2.3s p(95)=2.55s http_req_failed................: 0.00% ✓ 0 ✗ 1709 http_req_receiving.............: avg=1.24ms min=41.51µs med=128.22µs max=125.31ms p(90)=2.4ms p(95)=4.63ms http_req_sending...............: avg=167.16µs min=8.66µs med=20.41µs max=27.91ms p(90)=58.37µs p(95)=499.65µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=1.76s min=642.69ms med=1.71s max=3.44s p(90)=2.3s p(95)=2.55s http_reqs......................: 1709 54.997317/s iteration_duration.............: avg=1.79s min=652.84ms med=1.73s max=3.47s p(90)=2.33s p(95)=2.58s iterations.....................: 1709 54.997317/s vus............................: 20 min=20 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mercurius` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 4341 ✗ 0 data_received..................: 127 MB 4.1 MB/s data_sent......................: 1.7 MB 56 kB/s http_req_blocked...............: avg=64.5µs min=1.78µs med=3.59µs max=4.35ms p(90)=5.29µs p(95)=224.72µs http_req_connecting............: avg=54.77µs min=0s med=0s max=4.32ms p(90)=0s p(95)=183.56µs http_req_duration..............: avg=2.08s min=512.33ms med=2.06s max=4.3s p(90)=2.53s p(95)=2.72s { expected_response:true }...: avg=2.08s min=512.33ms med=2.06s max=4.3s p(90)=2.53s p(95)=2.72s http_req_failed................: 0.00% ✓ 0 ✗ 1447 http_req_receiving.............: avg=180.8µs min=41.22µs med=109.89µs max=7.98ms p(90)=220.71µs p(95)=338.6µs http_req_sending...............: avg=54.18µs min=9.12µs med=20.35µs max=6.94ms p(90)=37.02µs p(95)=144.44µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=2.08s min=511.78ms med=2.06s max=4.3s p(90)=2.53s p(95)=2.72s http_reqs......................: 1447 47.198003/s iteration_duration.............: avg=2.1s min=523.4ms med=2.06s max=4.32s p(90)=2.56s p(95)=2.73s iterations.....................: 1447 47.198003/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
github-actions[bot] commented 4 days ago

Overview for: federation-v1/constant-vus-subgraphs-delay

This scenario runs 4 subgraphs and a GraphQL gateway with Federation v1 spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 100 VUs over 30s

Comparison

Comparison
Gateway RPS ⬇️ Requests Duration Notes
cosmo 192 5855 total, 0 failed avg: 340ms, p95: 808ms ✅
apollo-router 173 5287 total, 0 failed avg: 482ms, p95: 702ms ✅
wundergraph 154 4717 total, 0 failed avg: 529ms, p95: 957ms ✅
mesh-supergraph-bun 77 2415 total, 0 failed avg: 1244ms, p95: 2173ms ✅
apollo-server 53 1689 total, 0 failed avg: 1811ms, p95: 2249ms ✅
mercurius 27 858 total, 0 failed avg: 3532ms, p95: 5342ms ✅
mesh-supergraph 23 743 total, 0 failed avg: 4179ms, p95: 6582ms ✅
Summary for: `cosmo` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 17565 ✗ 0 data_received..................: 514 MB 17 MB/s data_sent......................: 6.9 MB 229 kB/s http_req_blocked...............: avg=142.12µs min=1.26µs med=2.71µs max=238.9ms p(90)=4.06µs p(95)=5.01µs http_req_connecting............: avg=7.26µs min=0s med=0s max=2.79ms p(90)=0s p(95)=0s http_req_duration..............: avg=340.39ms min=8.35ms med=289.09ms max=2.46s p(90)=653.37ms p(95)=808.34ms { expected_response:true }...: avg=340.39ms min=8.35ms med=289.09ms max=2.46s p(90)=653.37ms p(95)=808.34ms http_req_failed................: 0.00% ✓ 0 ✗ 5855 http_req_receiving.............: avg=102.4ms min=31.71µs med=82.49µs max=2.43s p(90)=441.02ms p(95)=584.67ms http_req_sending...............: avg=5.04ms min=8.17µs med=13.23µs max=1.19s p(90)=41.97µs p(95)=439.3µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=232.95ms min=8.27ms med=210.15ms max=777.13ms p(90)=424.23ms p(95)=499.52ms http_reqs......................: 5855 192.769985/s iteration_duration.............: avg=516.31ms min=18.55ms med=442.79ms max=2.9s p(90)=1.01s p(95)=1.22s iterations.....................: 5855 192.769985/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `apollo-router` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 15861 ✗ 0 data_received..................: 464 MB 15 MB/s data_sent......................: 6.3 MB 206 kB/s http_req_blocked...............: avg=29.06µs min=1.51µs med=2.94µs max=7.81ms p(90)=4.63µs p(95)=5.84µs http_req_connecting............: avg=21.26µs min=0s med=0s max=6.44ms p(90)=0s p(95)=0s http_req_duration..............: avg=482.38ms min=11.32ms med=485.21ms max=1.67s p(90)=631.02ms p(95)=701.82ms { expected_response:true }...: avg=482.38ms min=11.32ms med=485.21ms max=1.67s p(90)=631.02ms p(95)=701.82ms http_req_failed................: 0.00% ✓ 0 ✗ 5287 http_req_receiving.............: avg=28.2ms min=35.38µs med=72.54µs max=1.33s p(90)=60.63ms p(95)=163.33ms http_req_sending...............: avg=1.32ms min=6.79µs med=13.81µs max=792ms p(90)=39.04µs p(95)=159.72µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=452.85ms min=11.21ms med=472.04ms max=833.3ms p(90)=590.98ms p(95)=626.52ms http_reqs......................: 5287 173.885457/s iteration_duration.............: avg=571.71ms min=36.48ms med=537.57ms max=2.14s p(90)=791.62ms p(95)=959.1ms iterations.....................: 5287 173.885457/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `wundergraph` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 14151 ✗ 0 data_received..................: 414 MB 14 MB/s data_sent......................: 5.6 MB 183 kB/s http_req_blocked...............: avg=126.6µs min=1.23µs med=2.85µs max=188.62ms p(90)=4.5µs p(95)=8.07µs http_req_connecting............: avg=85.68µs min=0s med=0s max=182.51ms p(90)=0s p(95)=0s http_req_duration..............: avg=528.67ms min=15.99ms med=496.84ms max=1.82s p(90)=826.78ms p(95)=957.05ms { expected_response:true }...: avg=528.67ms min=15.99ms med=496.84ms max=1.82s p(90)=826.78ms p(95)=957.05ms http_req_failed................: 0.00% ✓ 0 ✗ 4717 http_req_receiving.............: avg=45.69ms min=35.29µs med=80.46µs max=1.53s p(90)=124.81ms p(95)=337.09ms http_req_sending...............: avg=3.2ms min=7.34µs med=13.33µs max=955.38ms p(90)=49.47µs p(95)=617.83µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=479.77ms min=15ms med=458.75ms max=1.17s p(90)=771.16ms p(95)=881.12ms http_reqs......................: 4717 154.470588/s iteration_duration.............: avg=643.63ms min=38.31ms med=589.62ms max=3.36s p(90)=1.02s p(95)=1.21s iterations.....................: 4717 154.470588/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mesh-supergraph-bun` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 7245 ✗ 0 data_received..................: 212 MB 6.8 MB/s data_sent......................: 2.9 MB 93 kB/s http_req_blocked...............: avg=56.66µs min=1.34µs med=3.23µs max=4.27ms p(90)=5.4µs p(95)=13.68µs http_req_connecting............: avg=48.2µs min=0s med=0s max=4.23ms p(90)=0s p(95)=0s http_req_duration..............: avg=1.24s min=579.92ms med=1.06s max=3.24s p(90)=1.99s p(95)=2.17s { expected_response:true }...: avg=1.24s min=579.92ms med=1.06s max=3.24s p(90)=1.99s p(95)=2.17s http_req_failed................: 0.00% ✓ 0 ✗ 2415 http_req_receiving.............: avg=14.01ms min=39.29µs med=110.05µs max=587.46ms p(90)=3.51ms p(95)=91.17ms http_req_sending...............: avg=335µs min=8.17µs med=15.26µs max=85.53ms p(90)=36.67µs p(95)=130.72µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=1.22s min=579.71ms med=1.05s max=3.24s p(90)=1.98s p(95)=2.15s http_reqs......................: 2415 77.957558/s iteration_duration.............: avg=1.27s min=589.63ms med=1.09s max=3.27s p(90)=2.01s p(95)=2.2s iterations.....................: 2415 77.957558/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `apollo-server` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 5067 ✗ 0 data_received..................: 149 MB 4.7 MB/s data_sent......................: 2.0 MB 64 kB/s http_req_blocked...............: avg=156.11µs min=1.41µs med=3.51µs max=11.33ms p(90)=5.46µs p(95)=189.78µs http_req_connecting............: avg=76.35µs min=0s med=0s max=5.33ms p(90)=0s p(95)=94.74µs http_req_duration..............: avg=1.81s min=388.85ms med=1.53s max=19.81s p(90)=1.86s p(95)=2.24s { expected_response:true }...: avg=1.81s min=388.85ms med=1.53s max=19.81s p(90)=1.86s p(95)=2.24s http_req_failed................: 0.00% ✓ 0 ✗ 1689 http_req_receiving.............: avg=258.75µs min=48.8µs med=124.4µs max=55.59ms p(90)=193.57µs p(95)=270.6µs http_req_sending...............: avg=53.83µs min=8.61µs med=19.26µs max=8.45ms p(90)=36.15µs p(95)=94.32µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=1.81s min=388.52ms med=1.53s max=19.81s p(90)=1.86s p(95)=2.24s http_reqs......................: 1689 53.967064/s iteration_duration.............: avg=1.82s min=400.81ms med=1.54s max=19.82s p(90)=1.87s p(95)=2.25s iterations.....................: 1689 53.967064/s vus............................: 43 min=43 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mercurius` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 2574 ✗ 0 data_received..................: 75 MB 2.5 MB/s data_sent......................: 1.0 MB 33 kB/s http_req_blocked...............: avg=197.24µs min=1.91µs med=4.51µs max=8.12ms p(90)=315.75µs p(95)=1.46ms http_req_connecting............: avg=166.26µs min=0s med=0s max=4.14ms p(90)=218.43µs p(95)=1.37ms http_req_duration..............: avg=3.53s min=510.03ms med=3.31s max=8.18s p(90)=4.46s p(95)=5.34s { expected_response:true }...: avg=3.53s min=510.03ms med=3.31s max=8.18s p(90)=4.46s p(95)=5.34s http_req_failed................: 0.00% ✓ 0 ✗ 858 http_req_receiving.............: avg=654.99µs min=46.89µs med=119.14µs max=89.43ms p(90)=205.5µs p(95)=406.17µs http_req_sending...............: avg=110.72µs min=9.25µs med=24.62µs max=3.21ms p(90)=103.26µs p(95)=724.2µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=3.53s min=509.72ms med=3.31s max=8.18s p(90)=4.46s p(95)=5.34s http_reqs......................: 858 27.916776/s iteration_duration.............: avg=3.54s min=522.01ms med=3.32s max=8.19s p(90)=4.47s p(95)=5.35s iterations.....................: 858 27.916776/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mesh-supergraph` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 2229 ✗ 0 data_received..................: 65 MB 2.0 MB/s data_sent......................: 882 kB 28 kB/s http_req_blocked...............: avg=239.59µs min=1.74µs med=4.56µs max=7.33ms p(90)=517.67µs p(95)=1.53ms http_req_connecting............: avg=222.71µs min=0s med=0s max=6.71ms p(90)=399.47µs p(95)=1.48ms http_req_duration..............: avg=4.17s min=1.15s med=4.2s max=8.34s p(90)=6.04s p(95)=6.58s { expected_response:true }...: avg=4.17s min=1.15s med=4.2s max=8.34s p(90)=6.04s p(95)=6.58s http_req_failed................: 0.00% ✓ 0 ✗ 743 http_req_receiving.............: avg=1.66ms min=47.19µs med=171.11µs max=83.32ms p(90)=2.04ms p(95)=3.34ms http_req_sending...............: avg=113.15µs min=9.39µs med=26.1µs max=3.54ms p(90)=190.08µs p(95)=391.58µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=4.17s min=1.15s med=4.2s max=8.33s p(90)=6.04s p(95)=6.58s http_reqs......................: 743 23.203544/s iteration_duration.............: avg=4.19s min=1.17s med=4.21s max=8.35s p(90)=6.05s p(95)=6.59s iterations.....................: 743 23.203544/s vus............................: 8 min=8 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
github-actions[bot] commented 4 days ago

Overview for: federation-v1/constant-vus-over-time

This scenario runs 4 subgraphs and a GraphQL gateway with Federation v1 spec, and runs a heavy query. It's being executed with a constant amount of VUs over a fixed amount of time. It measure things like memory usage, CPU usage, average RPS. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was running 100 VUs over 30s

Comparison

Comparison
Gateway RPS ⬇️ Requests Duration Notes
cosmo 191 5808 total, 0 failed avg: 342ms, p95: 806ms ✅
apollo-router 175 5344 total, 0 failed avg: 471ms, p95: 690ms ✅
wundergraph 155 4734 total, 0 failed avg: 528ms, p95: 915ms ✅
mesh-supergraph-bun 70 2208 total, 0 failed avg: 1372ms, p95: 2392ms ✅
apollo-server 56 1758 total, 0 failed avg: 1735ms, p95: 1963ms ✅
mercurius 28 901 total, 0 failed avg: 3391ms, p95: 4605ms ✅
mesh-supergraph 25 814 total, 0 failed avg: 3832ms, p95: 6388ms ✅
Summary for: `cosmo` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 17424 ✗ 0 data_received..................: 510 MB 17 MB/s data_sent......................: 6.9 MB 228 kB/s http_req_blocked...............: avg=144.69µs min=1.48µs med=3.11µs max=220.9ms p(90)=4.99µs p(95)=6.06µs http_req_connecting............: avg=112.33µs min=0s med=0s max=220.84ms p(90)=0s p(95)=0s http_req_duration..............: avg=342.27ms min=10.66ms med=289.28ms max=2.18s p(90)=616.24ms p(95)=805.52ms { expected_response:true }...: avg=342.27ms min=10.66ms med=289.28ms max=2.18s p(90)=616.24ms p(95)=805.52ms http_req_failed................: 0.00% ✓ 0 ✗ 5808 http_req_receiving.............: avg=86.58ms min=35.11µs med=81.54µs max=1.8s p(90)=368.97ms p(95)=530.04ms http_req_sending...............: avg=5.37ms min=8.31µs med=14.33µs max=1.26s p(90)=66.57µs p(95)=2.29ms http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=250.31ms min=10.47ms med=238.7ms max=918.23ms p(90)=413.25ms p(95)=483.87ms http_reqs......................: 5808 191.804336/s iteration_duration.............: avg=518.88ms min=24.19ms med=448.24ms max=3.75s p(90)=968.55ms p(95)=1.21s iterations.....................: 5808 191.804336/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `apollo-router` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 16032 ✗ 0 data_received..................: 469 MB 15 MB/s data_sent......................: 6.3 MB 208 kB/s http_req_blocked...............: avg=25.39µs min=1.34µs med=2.82µs max=27.46ms p(90)=4.24µs p(95)=5.51µs http_req_connecting............: avg=14.98µs min=0s med=0s max=2.84ms p(90)=0s p(95)=0s http_req_duration..............: avg=471.15ms min=23.27ms med=469.91ms max=1.9s p(90)=620.94ms p(95)=689.81ms { expected_response:true }...: avg=471.15ms min=23.27ms med=469.91ms max=1.9s p(90)=620.94ms p(95)=689.81ms http_req_failed................: 0.00% ✓ 0 ✗ 5344 http_req_receiving.............: avg=37.84ms min=36.18µs med=70.16µs max=1.45s p(90)=80.34ms p(95)=285.03ms http_req_sending...............: avg=1.75ms min=8.1µs med=13.51µs max=640.67ms p(90)=50.58µs p(95)=811.46µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=431.55ms min=23.21ms med=452.38ms max=808.62ms p(90)=581.16ms p(95)=615.93ms http_reqs......................: 5344 175.39325/s iteration_duration.............: avg=566.67ms min=43.88ms med=525.95ms max=3.01s p(90)=781.73ms p(95)=1.02s iterations.....................: 5344 175.39325/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `wundergraph` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 14202 ✗ 0 data_received..................: 415 MB 14 MB/s data_sent......................: 5.6 MB 184 kB/s http_req_blocked...............: avg=206.71µs min=1.3µs med=2.76µs max=258.11ms p(90)=4.13µs p(95)=5.45µs http_req_connecting............: avg=129.12µs min=0s med=0s max=258.05ms p(90)=0s p(95)=0s http_req_duration..............: avg=528.41ms min=22.07ms med=503.29ms max=1.66s p(90)=805.76ms p(95)=915.24ms { expected_response:true }...: avg=528.41ms min=22.07ms med=503.29ms max=1.66s p(90)=805.76ms p(95)=915.24ms http_req_failed................: 0.00% ✓ 0 ✗ 4734 http_req_receiving.............: avg=30.23ms min=38.15µs med=75.77µs max=1.47s p(90)=93.12ms p(95)=195.97ms http_req_sending...............: avg=2.3ms min=7.96µs med=13.34µs max=370.9ms p(90)=54.81µs p(95)=1.52ms http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=495.87ms min=22ms med=479.42ms max=1.29s p(90)=765.66ms p(95)=864.83ms http_reqs......................: 4734 155.009635/s iteration_duration.............: avg=640.88ms min=48.01ms med=602.34ms max=2.2s p(90)=987.21ms p(95)=1.13s iterations.....................: 4734 155.009635/s vus............................: 100 min=100 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mesh-supergraph-bun` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 6624 ✗ 0 data_received..................: 194 MB 6.2 MB/s data_sent......................: 2.6 MB 84 kB/s http_req_blocked...............: avg=112.04µs min=1.41µs med=3.12µs max=67ms p(90)=5.4µs p(95)=23.64µs http_req_connecting............: avg=73.53µs min=0s med=0s max=4.75ms p(90)=0s p(95)=0s http_req_duration..............: avg=1.37s min=673.24ms med=1.18s max=3.02s p(90)=2.22s p(95)=2.39s { expected_response:true }...: avg=1.37s min=673.24ms med=1.18s max=3.02s p(90)=2.22s p(95)=2.39s http_req_failed................: 0.00% ✓ 0 ✗ 2208 http_req_receiving.............: avg=14.03ms min=41.08µs med=120.39µs max=600.91ms p(90)=2.22ms p(95)=41.47ms http_req_sending...............: avg=185.29µs min=8.45µs med=15.16µs max=29.21ms p(90)=42.56µs p(95)=171.94µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=1.35s min=672.65ms med=1.17s max=3.01s p(90)=2.22s p(95)=2.38s http_reqs......................: 2208 70.716281/s iteration_duration.............: avg=1.39s min=686.92ms med=1.2s max=3.09s p(90)=2.27s p(95)=2.41s iterations.....................: 2208 70.716281/s vus............................: 19 min=19 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `apollo-server` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 5274 ✗ 0 data_received..................: 155 MB 5.0 MB/s data_sent......................: 2.1 MB 67 kB/s http_req_blocked...............: avg=83.19µs min=1.38µs med=2.68µs max=5.24ms p(90)=4.54µs p(95)=171.66µs http_req_connecting............: avg=73.95µs min=0s med=0s max=3.03ms p(90)=0s p(95)=142.24µs http_req_duration..............: avg=1.73s min=696.47ms med=1.46s max=18.43s p(90)=1.76s p(95)=1.96s { expected_response:true }...: avg=1.73s min=696.47ms med=1.46s max=18.43s p(90)=1.76s p(95)=1.96s http_req_failed................: 0.00% ✓ 0 ✗ 1758 http_req_receiving.............: avg=163.55µs min=42.96µs med=114.08µs max=22.26ms p(90)=180.77µs p(95)=239.84µs http_req_sending...............: avg=47.61µs min=8.16µs med=14.08µs max=5.1ms p(90)=31.05µs p(95)=93.25µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=1.73s min=696.28ms med=1.46s max=18.43s p(90)=1.76s p(95)=1.96s http_reqs......................: 1758 56.350144/s iteration_duration.............: avg=1.74s min=711.62ms med=1.47s max=18.43s p(90)=1.78s p(95)=1.97s iterations.....................: 1758 56.350144/s vus............................: 34 min=34 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mercurius` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 2703 ✗ 0 data_received..................: 79 MB 2.5 MB/s data_sent......................: 1.1 MB 34 kB/s http_req_blocked...............: avg=151.73µs min=1.72µs med=3.8µs max=4.93ms p(90)=179.45µs p(95)=1.28ms http_req_connecting............: avg=134.03µs min=0s med=0s max=4.86ms p(90)=119.25µs p(95)=1.1ms http_req_duration..............: avg=3.39s min=652.73ms med=3.19s max=7.6s p(90)=4.25s p(95)=4.6s { expected_response:true }...: avg=3.39s min=652.73ms med=3.19s max=7.6s p(90)=4.25s p(95)=4.6s http_req_failed................: 0.00% ✓ 0 ✗ 901 http_req_receiving.............: avg=378.19µs min=41.88µs med=113.78µs max=37.01ms p(90)=191.4µs p(95)=289.09µs http_req_sending...............: avg=113.63µs min=9.82µs med=21.56µs max=13.39ms p(90)=55.29µs p(95)=483.05µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=3.39s min=652.31ms med=3.19s max=7.6s p(90)=4.25s p(95)=4.6s http_reqs......................: 901 28.943408/s iteration_duration.............: avg=3.4s min=661.11ms med=3.2s max=7.62s p(90)=4.26s p(95)=4.61s iterations.....................: 901 28.943408/s vus............................: 26 min=26 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mesh-supergraph` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 2442 ✗ 0 data_received..................: 71 MB 2.2 MB/s data_sent......................: 966 kB 30 kB/s http_req_blocked...............: avg=80.31µs min=1.36µs med=3.58µs max=3.88ms p(90)=147.87µs p(95)=535.91µs http_req_connecting............: avg=68.76µs min=0s med=0s max=3.86ms p(90)=114.87µs p(95)=490.08µs http_req_duration..............: avg=3.83s min=1.28s med=3.86s max=7.6s p(90)=5.6s p(95)=6.38s { expected_response:true }...: avg=3.83s min=1.28s med=3.86s max=7.6s p(90)=5.6s p(95)=6.38s http_req_failed................: 0.00% ✓ 0 ✗ 814 http_req_receiving.............: avg=1.48ms min=40.57µs med=137.4µs max=71.18ms p(90)=1.99ms p(95)=4.04ms http_req_sending...............: avg=60.01µs min=7.97µs med=20.8µs max=8.69ms p(90)=97.29µs p(95)=248.68µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=3.83s min=1.27s med=3.86s max=7.6s p(90)=5.6s p(95)=6.38s http_reqs......................: 814 25.426858/s iteration_duration.............: avg=3.84s min=1.29s med=3.88s max=7.61s p(90)=5.61s p(95)=6.44s iterations.....................: 814 25.426858/s vus............................: 5 min=5 max=100 vus_max........................: 100 min=100 max=100 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
github-actions[bot] commented 4 days ago

Overview for: federation-v1/ramping-vus

This scenario runs 4 subgraphs and a GraphQL gateway with Federation v1 spec, and runs a heavy query. We are running a heavy load of concurrent VUs to measure response time and other stats, during stress. It measure things like memory usage, CPU usage, response times. It also includes a summary of the entire execution, and metrics information about HTTP execution times.

This scenario was trying to reach 500 concurrent VUs over 60s

Comparison

Comparison
Gateway duration(p95)⬇️ RPS Requests Durations Notes
cosmo 2044ms 183 12859 total, 0 failed avg: 743ms, p95: 2045ms, max: 5531ms, med: 545ms ✅
wundergraph 2236ms 159 11132 total, 0 failed avg: 785ms, p95: 2236ms, max: 5386ms, med: 599ms ✅
apollo-router 2742ms 164 11530 total, 0 failed avg: 884ms, p95: 2743ms, max: 8032ms, med: 586ms ✅
mesh-supergraph-bun 7304ms 87 6130 total, 0 failed avg: 3066ms, p95: 7305ms, max: 12386ms, med: 2813ms ✅
mesh-supergraph 8061ms 57 4006 total, 0 failed avg: 4828ms, p95: 8061ms, max: 10331ms, med: 4972ms ✅
apollo-server 21095ms 73 5357 total, 0 failed avg: 3866ms, p95: 21096ms, max: 43708ms, med: 2124ms ✅
Summary for: `cosmo` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 38577 ✗ 0 data_received..................: 1.1 GB 16 MB/s data_sent......................: 15 MB 218 kB/s http_req_blocked...............: avg=19.45ms min=1.26µs med=3.13µs max=3.96s p(90)=5.06µs p(95)=17.42µs http_req_connecting............: avg=18.31ms min=0s med=0s max=2.53s p(90)=0s p(95)=0s http_req_duration..............: avg=743.19ms min=3.81ms med=545.24ms max=5.53s p(90)=1.69s p(95)=2.04s { expected_response:true }...: avg=743.19ms min=3.81ms med=545.24ms max=5.53s p(90)=1.69s p(95)=2.04s http_req_failed................: 0.00% ✓ 0 ✗ 12859 http_req_receiving.............: avg=211.73ms min=34µs med=81.31µs max=4.67s p(90)=895.34ms p(95)=1.29s http_req_sending...............: avg=30.98ms min=8.33µs med=15.14µs max=4.27s p(90)=520.77µs p(95)=102.25ms http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=500.47ms min=3.74ms med=382.57ms max=3.07s p(90)=1.12s p(95)=1.32s http_reqs......................: 12859 183.670826/s iteration_duration.............: avg=1.42s min=9.32ms med=1.07s max=10.1s p(90)=3.22s p(95)=4.01s iterations.....................: 12859 183.670826/s vus............................: 1 min=1 max=499 vus_max........................: 500 min=500 max=500 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `wundergraph` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 33396 ✗ 0 data_received..................: 977 MB 14 MB/s data_sent......................: 13 MB 189 kB/s http_req_blocked...............: avg=20.82ms min=1.5µs med=3.31µs max=3.36s p(90)=5.44µs p(95)=30.56µs http_req_connecting............: avg=19.86ms min=0s med=0s max=3.36s p(90)=0s p(95)=0s http_req_duration..............: avg=785.16ms min=6.15ms med=598.64ms max=5.38s p(90)=1.75s p(95)=2.23s { expected_response:true }...: avg=785.16ms min=6.15ms med=598.64ms max=5.38s p(90)=1.75s p(95)=2.23s http_req_failed................: 0.00% ✓ 0 ✗ 11132 http_req_receiving.............: avg=224.18ms min=32.57µs med=84.79µs max=4.51s p(90)=971.49ms p(95)=1.42s http_req_sending...............: avg=37.49ms min=8.24µs med=15.32µs max=3.61s p(90)=1.14ms p(95)=145.58ms http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=523.48ms min=6.08ms med=437.17ms max=2.87s p(90)=1.06s p(95)=1.27s http_reqs......................: 11132 159.020542/s iteration_duration.............: avg=1.64s min=12.78ms med=1.22s max=10.94s p(90)=3.69s p(95)=4.57s iterations.....................: 11132 159.020542/s vus............................: 1 min=1 max=499 vus_max........................: 500 min=500 max=500 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `apollo-router` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 34590 ✗ 0 data_received..................: 1.0 GB 14 MB/s data_sent......................: 14 MB 196 kB/s http_req_blocked...............: avg=25.16ms min=1.63µs med=3.34µs max=5.37s p(90)=5.75µs p(95)=23.85µs http_req_connecting............: avg=22.33ms min=0s med=0s max=5.37s p(90)=0s p(95)=0s http_req_duration..............: avg=884.25ms min=6.08ms med=585.85ms max=8.03s p(90)=1.98s p(95)=2.74s { expected_response:true }...: avg=884.25ms min=6.08ms med=585.85ms max=8.03s p(90)=1.98s p(95)=2.74s http_req_failed................: 0.00% ✓ 0 ✗ 11530 http_req_receiving.............: avg=348.77ms min=33.21µs med=96.61µs max=6.05s p(90)=1.23s p(95)=1.92s http_req_sending...............: avg=35.78ms min=8.11µs med=15.61µs max=5.51s p(90)=176.79µs p(95)=70.02ms http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=499.69ms min=5.99ms med=395.63ms max=3.19s p(90)=1.05s p(95)=1.29s http_reqs......................: 11530 164.704345/s iteration_duration.............: avg=1.62s min=11.62ms med=1.09s max=15.03s p(90)=3.81s p(95)=4.85s iterations.....................: 11530 164.704345/s vus............................: 1 min=1 max=499 vus_max........................: 500 min=500 max=500 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mesh-supergraph-bun` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 18390 ✗ 0 data_received..................: 538 MB 7.7 MB/s data_sent......................: 7.3 MB 104 kB/s http_req_blocked...............: avg=1.95ms min=1.59µs med=3µs max=349.68ms p(90)=9.47µs p(95)=242.28µs http_req_connecting............: avg=1.91ms min=0s med=0s max=349.62ms p(90)=0s p(95)=164.77µs http_req_duration..............: avg=3.06s min=8.53ms med=2.81s max=12.38s p(90)=5.25s p(95)=7.3s { expected_response:true }...: avg=3.06s min=8.53ms med=2.81s max=12.38s p(90)=5.25s p(95)=7.3s http_req_failed................: 0.00% ✓ 0 ✗ 6130 http_req_receiving.............: avg=29.35ms min=33µs med=77.41µs max=1.85s p(90)=2.67ms p(95)=137.17ms http_req_sending...............: avg=2.48ms min=8.97µs med=14.22µs max=852.27ms p(90)=118.41µs p(95)=11.1ms http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=3.03s min=8.34ms med=2.76s max=12.36s p(90)=5.23s p(95)=7.3s http_reqs......................: 6130 87.563026/s iteration_duration.............: avg=3.13s min=13.96ms med=2.89s max=12.61s p(90)=5.29s p(95)=7.4s iterations.....................: 6130 87.563026/s vus............................: 1 min=1 max=500 vus_max........................: 500 min=500 max=500 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `mesh-supergraph` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 12018 ✗ 0 data_received..................: 352 MB 5.0 MB/s data_sent......................: 4.8 MB 68 kB/s http_req_blocked...............: avg=347.1µs min=1.3µs med=3.82µs max=107.59ms p(90)=186.91µs p(95)=448.33µs http_req_connecting............: avg=327.48µs min=0s med=0s max=107.37ms p(90)=129.67µs p(95)=367.99µs http_req_duration..............: avg=4.82s min=12.81ms med=4.97s max=10.33s p(90)=7.56s p(95)=8.06s { expected_response:true }...: avg=4.82s min=12.81ms med=4.97s max=10.33s p(90)=7.56s p(95)=8.06s http_req_failed................: 0.00% ✓ 0 ✗ 4006 http_req_receiving.............: avg=1.3ms min=37.66µs med=120.29µs max=153.72ms p(90)=2.22ms p(95)=4.23ms http_req_sending...............: avg=339.95µs min=8.19µs med=21.84µs max=110.33ms p(90)=52.43µs p(95)=98.99µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=4.82s min=12.7ms med=4.96s max=10.32s p(90)=7.56s p(95)=8.05s http_reqs......................: 4006 57.225208/s iteration_duration.............: avg=4.85s min=19.33ms med=4.99s max=10.55s p(90)=7.59s p(95)=8.07s iterations.....................: 4006 57.225208/s vus............................: 1 min=1 max=499 vus_max........................: 500 min=500 max=500 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview
Summary for: `apollo-server` **K6 Output** ``` ✓ response code was 200 ✓ no graphql errors ✓ valid response structure checks.........................: 100.00% ✓ 16071 ✗ 0 data_received..................: 471 MB 6.5 MB/s data_sent......................: 6.4 MB 87 kB/s http_req_blocked...............: avg=98.16µs min=1.47µs med=3.13µs max=37.49ms p(90)=13.92µs p(95)=228.94µs http_req_connecting............: avg=88.37µs min=0s med=0s max=37.4ms p(90)=0s p(95)=156.55µs http_req_duration..............: avg=3.86s min=58.62ms med=2.12s max=43.7s p(90)=3.1s p(95)=21.09s { expected_response:true }...: avg=3.86s min=58.62ms med=2.12s max=43.7s p(90)=3.1s p(95)=21.09s http_req_failed................: 0.00% ✓ 0 ✗ 5357 http_req_receiving.............: avg=210.78µs min=44.8µs med=113.41µs max=134.57ms p(90)=194.46µs p(95)=281.92µs http_req_sending...............: avg=50.41µs min=8.55µs med=16.45µs max=25.55ms p(90)=38.96µs p(95)=56.89µs http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s http_req_waiting...............: avg=3.86s min=58.52ms med=2.12s max=43.7s p(90)=3.1s p(95)=21.09s http_reqs......................: 5357 73.596031/s iteration_duration.............: avg=3.87s min=69.21ms med=2.13s max=43.75s p(90)=3.11s p(95)=21.1s iterations.....................: 5357 73.596031/s vus............................: 61 min=57 max=500 vus_max........................: 500 min=500 max=500 ``` **Performance Overview** Performance Overview **Subgraphs Overview** Subgraphs Overview **HTTP Overview** HTTP Overview