Closed viniarck closed 1 year ago
Here's an update on this issue:
Good news:
Flask
SocketIO
server with async_mode=threading
(that I'll share the results below), is able to handle hundreds of request per second, with based on the current NApps should be plenty. eventlent
and gevent
doesn't require much code changes.Heads up:
eventlet
or gevent
we need to also be aware that since green threads patch certain methods, and will affect the behavior of the decorated handlers, as Miguel Grinberg has highlighted and explained on this issue and on this comment, and some DB drivers might have problems with that, so before selecting this I think we should only move forward with this once we have selected a new DB we'll support to be used by the NApps, so I think this is pre-requisite to ensure a smooth transition. gunicorn
, and also the fact that kytos has a shell prompt that owns the main thread, and NApps also assume they'd be in the same process, so both gevent
or eventlet
or any other async server (and or more async def usage) would be easier to integrate in the current architecture.The benchmarks were run against two endpoints GET kytos/topology/v3
and POST /flow_manager/v2/flows
using both the current theading
async_mode (threaded without upper bound limit), and the second mode was eventlet
. Keep in mind the point isn't to try to find out the highest number it can handle, but instead try to make sure it works fine handling hundreds of requests/sec, the benchmarks were run on my computer i7-9750H CPU @ 2.60GHz, 16GB RAM+:
threading
async_mode❯ echo "GET http://localhost:8181/api/kytos/topology/v3" | vegeta attack -rate 100/1s -duration=120s | tee results.bin | vegeta report
Requests [total, rate, throughput] 12000, 100.01, 100.00
Duration [total, attack, wait] 2m0s, 2m0s, 7.288ms
Latencies [min, mean, 50, 90, 95, 99, max] 1.354ms, 7.088ms, 6.701ms, 7.399ms, 8.007ms, 32.238ms, 148.949ms
Bytes In [total, mean] 31716000, 2643.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:12000
Error Set:
❯ echo "GET http://localhost:8181/api/kytos/topology/v3" | vegeta attack -rate 200/1s -duration=120s | tee results.bin | vegeta report
Requests [total, rate, throughput] 24000, 200.01, 199.81
Duration [total, attack, wait] 2m0s, 2m0s, 116.094ms
Latencies [min, mean, 50, 90, 95, 99, max] 1.305ms, 338.282ms, 204.913ms, 863.765ms, 1.006s, 1.502s, 3.735s
Bytes In [total, mean] 63432000, 2643.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:24000
Error Set:
❯ jq -ncM '{method: "POST", url: "http://localhost:8181/api/kytos/flow_manager/v2/flows/00:00:00:00:00:00:00:01", body: { "force": true, "flows": [ { "priority": 10, "match": { "in_port"
: 1, "dl_vlan": 100 }, "actions": [ { "action_type": "output", "port": 1 } ] } ] } | @base64, header: {"Content-Type": ["application/json"]}}' | vegeta attack -format=json -rate 100/1s -
duration=120s | tee results.bin | vegeta report
Requests [total, rate, throughput] 12000, 100.01, 100.00
Duration [total, attack, wait] 2m0s, 2m0s, 11.48ms
Latencies [min, mean, 50, 90, 95, 99, max] 2.12ms, 82.421ms, 17.091ms, 274.818ms, 527.182ms, 877.957ms, 1.553s
Bytes In [total, mean] 444000, 37.00
Bytes Out [total, mean] 1464000, 122.00
Success [ratio] 100.00%
Status Codes [code:count] 202:12000
Error Set:
eventlet==0.33.0
async_mode❯ echo "GET http://localhost:8181/api/kytos/topology/v3" | vegeta attack -rate 100/1s -duration=120s | tee results.bin | vegeta report
Requests [total, rate, throughput] 12000, 100.01, 100.00
Duration [total, attack, wait] 2m0s, 2m0s, 4.111ms
Latencies [min, mean, 50, 90, 95, 99, max] 1.091ms, 6.964ms, 5.203ms, 6.253ms, 8.758ms, 83.573ms, 211.519ms
Bytes In [total, mean] 106260000, 8855.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:12000
Error Set:
❯ echo "GET http://localhost:8181/api/kytos/topology/v3" | vegeta attack -rate 200/1s -duration=120s | tee results.bin | vegeta report
Requests [total, rate, throughput] 24000, 200.01, 198.14
Duration [total, attack, wait] 2m1s, 2m0s, 1.134s
Latencies [min, mean, 50, 90, 95, 99, max] 1.001s, 1.006s, 1.003s, 1.007s, 1.017s, 1.06s, 1.188s
Bytes In [total, mean] 212520000, 8855.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:24000
Error Set:
❯ jq -ncM '{method: "POST", url: "http://localhost:8181/api/kytos/flow_manager/v2/flows/00:00:00:00:00:00:00:01", body: { "force": true, "flows": [ { "priority": 10, "match": { "in_port"
: 1, "dl_vlan": 100 }, "actions": [ { "action_type": "output", "port": 1 } ] } ] } | @base64, header: {"Content-Type": ["application/json"]}}' | vegeta attack -format=json -rate 100/1s -
duration=120s | tee results.bin | vegeta report
Requests [total, rate, throughput] 12000, 100.01, 100.01
Duration [total, attack, wait] 2m0s, 2m0s, 2.13ms
Latencies [min, mean, 50, 90, 95, 99, max] 1.643ms, 14.446ms, 2.325ms, 29.73ms, 76.319ms, 232.193ms, 532.699ms
Bytes In [total, mean] 444000, 37.00
Bytes Out [total, mean] 1464000, 122.00
Success [ratio] 100.00%
Status Codes [code:count] 202:12000
Error Set:
As of now,
Flask
SocketIO
server instance usesFlask
development server, which works great, however as officially stated and recommend in their docs:flask-socketio
:I think we should assess shipping kytos with
gevent
oreventlet
as a depedency just so that way it's production-ready, and then operators could run in production out of the box, or alternatively also run withgunicorn