Benchmarks for graphql

[petr]

Benchmarks for our service[ BE]

We need to compare the 'code' performance of the graphene.

So we need a tool we can run and compare two methods classic HTTP api and grahpql.

The best way to compare is to use simple rest application (Flask or Falcon) that provides simple functionality. A couple of methods - for example get clans and accounts (/clans/ and /accounts/)

Using of docker compose file will be great solution for running tests

[frsv]

I create benchmark tests for expensive graphql query and for similar app which uses only requests and internal processing logic Each test tries to get info for 100 clans, each has 100 members in it. All fields in response are camelcased (it's required to add this to requests example for similarity of tests)

Using another docker-compose file for benchmarks is overhead. Furthermore, pytest-benchmark is part of pytest and benchmark tests easy to skip with --benchmark-skip option.

Also, use simple rest application with Falcon or Flask also not necessary, cause we have to test a time of requesting and collecting data, not efficiency of different python web frameworks integrations with GraphQL

So I've got results which seem not really efficient:

--------------------------------------------------------------------------------------------- benchmark: 2 tests ---------------------------------------------------------------------------------------------
Name (time in ms)                               Min                 Max                Mean            StdDev              Median                IQR            Outliers     OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_complex_request_benchmark              96.2461 (1.0)      120.8586 (1.0)      105.6474 (1.0)      8.5200 (1.0)      103.6510 (1.0)      13.0493 (1.0)           3;0  9.4654 (1.0)          10           1
test_complex_request_graphql_benchmark     376.3585 (3.91)     399.6960 (3.31)     390.6440 (3.70)     9.3990 (1.10)     393.1081 (3.79)     13.8735 (1.06)          1;0  2.5599 (0.27)          5           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------