API server Flask `SocketIO` should be ready for running in production out of the box

[viniarck]

As of now, Flask SocketIO server instance uses Flask development server, which works great, however as officially stated and recommend in their docs:

When running publicly rather than in development, you should not use the built-in development server (flask run). The development server is provided by Werkzeug for convenience, but is not designed to be particularly efficient, stable, or secure.

flask-socketio:

If eventlet or gevent are available, socketio.run(app) starts a production-ready server using one of these frameworks. If neither of these are installed, then the Flask development web server is used, and in this case the server is not intended to be used in a production deployment.

I think we should assess shipping kytos with gevent or eventlet as a depedency just so that way it's production-ready, and then operators could run in production out of the box, or alternatively also run with gunicorn

[viniarck]

Here's an update on this issue:

Good news:

• The benchmarks using the current Flask SocketIO server with async_mode=threading (that I'll share the results below), is able to handle hundreds of request per second, with based on the current NApps should be plenty.
• Adding support to eventlent and gevent doesn't require much code changes.

Heads up:

• To integrate either eventlet or gevent we need to also be aware that since green threads patch certain methods, and will affect the behavior of the decorated handlers, as Miguel Grinberg has highlighted and explained on this issue and on this comment, and some DB drivers might have problems with that, so before selecting this I think we should only move forward with this once we have selected a new DB we'll support to be used by the NApps, so I think this is pre-requisite to ensure a smooth transition.
• Currently, the API server assumes it's running in the same process, this makes it difficult to hook it with gunicorn, and also the fact that kytos has a shell prompt that owns the main thread, and NApps also assume they'd be in the same process, so both gevent or eventlet or any other async server (and or more async def usage) would be easier to integrate in the current architecture.

Benchmarks

The benchmarks were run against two endpoints GET kytos/topology/v3 and POST /flow_manager/v2/flows using both the current theading async_mode (threaded without upper bound limit), and the second mode was eventlet. Keep in mind the point isn't to try to find out the highest number it can handle, but instead try to make sure it works fine handling hundreds of requests/sec, the benchmarks were run on my computer i7-9750H CPU @ 2.60GHz, 16GB RAM+:

Current threading async_mode

• Notice the success rate was 100%, the latency times were also relatively great as the number of requests were increased up to 200 requests / sec, which is plenty considering the current workload we expect:

❯ echo "GET http://localhost:8181/api/kytos/topology/v3" | vegeta attack -rate 100/1s -duration=120s | tee results.bin | vegeta report
Requests      [total, rate, throughput]         12000, 100.01, 100.00
Duration      [total, attack, wait]             2m0s, 2m0s, 7.288ms
Latencies     [min, mean, 50, 90, 95, 99, max]  1.354ms, 7.088ms, 6.701ms, 7.399ms, 8.007ms, 32.238ms, 148.949ms
Bytes In      [total, mean]                     31716000, 2643.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:12000
Error Set:

❯ echo "GET http://localhost:8181/api/kytos/topology/v3" | vegeta attack -rate 200/1s -duration=120s | tee results.bin | vegeta report
Requests      [total, rate, throughput]         24000, 200.01, 199.81
Duration      [total, attack, wait]             2m0s, 2m0s, 116.094ms
Latencies     [min, mean, 50, 90, 95, 99, max]  1.305ms, 338.282ms, 204.913ms, 863.765ms, 1.006s, 1.502s, 3.735s
Bytes In      [total, mean]                     63432000, 2643.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:24000
Error Set:

❯ jq -ncM '{method: "POST", url: "http://localhost:8181/api/kytos/flow_manager/v2/flows/00:00:00:00:00:00:00:01", body: { "force": true, "flows": [ { "priority": 10, "match": { "in_port"
: 1, "dl_vlan": 100 }, "actions": [ { "action_type": "output", "port": 1 } ] } ] } | @base64, header: {"Content-Type": ["application/json"]}}' | vegeta attack -format=json -rate 100/1s -
duration=120s | tee results.bin | vegeta report
Requests      [total, rate, throughput]         12000, 100.01, 100.00
Duration      [total, attack, wait]             2m0s, 2m0s, 11.48ms
Latencies     [min, mean, 50, 90, 95, 99, max]  2.12ms, 82.421ms, 17.091ms, 274.818ms, 527.182ms, 877.957ms, 1.553s
Bytes In      [total, mean]                     444000, 37.00
Bytes Out     [total, mean]                     1464000, 122.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      202:12000
Error Set:

eventlet==0.33.0 async_mode

❯ echo "GET http://localhost:8181/api/kytos/topology/v3" | vegeta attack -rate 100/1s -duration=120s | tee results.bin | vegeta report
Requests      [total, rate, throughput]         12000, 100.01, 100.00
Duration      [total, attack, wait]             2m0s, 2m0s, 4.111ms
Latencies     [min, mean, 50, 90, 95, 99, max]  1.091ms, 6.964ms, 5.203ms, 6.253ms, 8.758ms, 83.573ms, 211.519ms
Bytes In      [total, mean]                     106260000, 8855.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:12000
Error Set:

❯ echo "GET http://localhost:8181/api/kytos/topology/v3" | vegeta attack -rate 200/1s -duration=120s | tee results.bin | vegeta report
Requests      [total, rate, throughput]         24000, 200.01, 198.14
Duration      [total, attack, wait]             2m1s, 2m0s, 1.134s
Latencies     [min, mean, 50, 90, 95, 99, max]  1.001s, 1.006s, 1.003s, 1.007s, 1.017s, 1.06s, 1.188s
Bytes In      [total, mean]                     212520000, 8855.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:24000
Error Set:

❯ jq -ncM '{method: "POST", url: "http://localhost:8181/api/kytos/flow_manager/v2/flows/00:00:00:00:00:00:00:01", body: { "force": true, "flows": [ { "priority": 10, "match": { "in_port"
: 1, "dl_vlan": 100 }, "actions": [ { "action_type": "output", "port": 1 } ] } ] } | @base64, header: {"Content-Type": ["application/json"]}}' | vegeta attack -format=json -rate 100/1s -
duration=120s | tee results.bin | vegeta report
Requests      [total, rate, throughput]         12000, 100.01, 100.01
Duration      [total, attack, wait]             2m0s, 2m0s, 2.13ms
Latencies     [min, mean, 50, 90, 95, 99, max]  1.643ms, 14.446ms, 2.325ms, 29.73ms, 76.319ms, 232.193ms, 532.699ms
Bytes In      [total, mean]                     444000, 37.00
Bytes Out     [total, mean]                     1464000, 122.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      202:12000
Error Set: