feat: added support for batching FlowMods

Closes #171

Summary

See updated changelog file
Version of the API wasn't bumped since it'll need to be bumped in the future for the rest of validations on v3, so we're saving a bit of refactoring on endpoint parametrization, since it's breaking compatibility it's a reasonable tradeoff for now.

Local Tests

Made a request with force: false with batching, it didn't block the request

2023-08-31 16:42:52,272 - INFO [kytos.napps.kytos/flow_manager] (AnyIO worker thread) Send FlowMod from request dpid: 00:00:00:00:00:00:00:01, command: add, force: False, batch_size: 2
, batch_interval: 5, flows[0, 3]: [{'priority': 4000, 'cookie': 100, 'match': {'in_port': 1, 'dl_vlan': 1}, 'actions': [{'action_type': 'output', 'port': 2}]}, {'priority': 4000, 'cook
ie': 100, 'match': {'in_port': 1, 'dl_vlan': 2}, 'actions': [{'action_type': 'output', 'port': 2}]}, {'priority': 4000, 'cookie': 100, 'match': {'in_port': 1, 'dl_vlan': 3}, 'actions':
 [{'action_type': 'output', 'port': 2}]}]
2023-08-31 16:42:52,281 - INFO [kytos.napps.kytos/flow_manager] (AnyIO worker thread) Sending FlowMods slice[0: 2], iteration: 0, flows[0, 2]: [{'switch': '00:00:00:00:00:00:00:01', 't
able_id': 0, 'match': {'in_port': 1, 'dl_vlan': 1}, 'priority': 4000, 'idle_timeout': 0, 'hard_timeout': 0, 'cookie': 100, 'id': '2e371cca3dcfe383a5058c4410920c1e', 'stats': {}, 'cooki
e_mask': 0, 'instructions': [{'instruction_type': 'apply_actions', 'actions': [{'port': 2, 'action_type': 'output'}]}]}, {'switch': '00:00:00:00:00:00:00:01', 'table_id': 0, 'match': {
'in_port': 1, 'dl_vlan': 2}, 'priority': 4000, 'idle_timeout': 0, 'hard_timeout': 0, 'cookie': 100, 'id': '8f4ebd12d31dba67b36dadfad7ba2c84', 'stats': {}, 'cookie_mask': 0, 'instructio
ns': [{'instruction_type': 'apply_actions', 'actions': [{'port': 2, 'action_type': 'output'}]}]}]
2023-08-31 16:42:52,282 - INFO [kytos.napps.kytos/flow_manager] (AnyIO worker thread) Send FlowMods batching will sleep for 5 seconds before sending slice[2:2]
kytos $>                                                                                                                                                                                

kytos $> 2023-08-31 16:42:57,285 - INFO [kytos.napps.kytos/flow_manager] (AnyIO worker thread) Sending FlowMods slice[2: 4], iteration: 1, flows[0, 1]: [{'switch': '00:00:00:00:00:00:0
0:01', 'table_id': 0, 'match': {'in_port': 1, 'dl_vlan': 3}, 'priority': 4000, 'idle_timeout': 0, 'hard_timeout': 0, 'cookie': 100, 'id': '6f614c8654066e827cce5dc6b3e785cb', 'stats': {
}, 'cookie_mask': 0, 'instructions': [{'instruction_type': 'apply_actions', 'actions': [{'port': 2, 'action_type': 'output'}]}]}]
2023-08-31 16:42:57,291 - INFO [uvicorn.access] (MainThread) 127.0.0.1:52294 - "POST /api/kytos/flow_manager/v2/flows/00%3A00%3A00%3A00%3A00%3A00%3A00%3A01 HTTP/1.1" 202

Made a request with force: true with batching, it blocked the request as expected

kytos $> 2023-08-31 16:43:14,817 - INFO [kytos.napps.kytos/flow_manager] (AnyIO worker thread) Send FlowMod from request dpid: 00:00:00:00:00:00:00:01, command: add, force: True, batch
_size: 2, batch_interval: 5, flows[0, 3]: [{'priority': 4000, 'cookie': 100, 'match': {'in_port': 1, 'dl_vlan': 1}, 'actions': [{'action_type': 'output', 'port': 2}]}, {'priority': 400
0, 'cookie': 100, 'match': {'in_port': 1, 'dl_vlan': 2}, 'actions': [{'action_type': 'output', 'port': 2}]}, {'priority': 4000, 'cookie': 100, 'match': {'in_port': 1, 'dl_vlan': 3}, 'a
ctions': [{'action_type': 'output', 'port': 2}]}]
2023-08-31 16:43:14,852 - INFO [uvicorn.access] (MainThread) 127.0.0.1:45808 - "POST /api/kytos/flow_manager/v2/flows/00%3A00%3A00%3A00%3A00%3A00%3A00%3A01 HTTP/1.1" 202
2023-08-31 16:43:14,857 - INFO [kytos.napps.kytos/flow_manager] (thread_pool_app_12) Sending FlowMods slice[0: 2], iteration: 0, flows[0, 2]: [{'switch': '00:00:00:00:00:00:00:01', 'ta
ble_id': 0, 'match': {'in_port': 1, 'dl_vlan': 1}, 'priority': 4000, 'idle_timeout': 0, 'hard_timeout': 0, 'cookie': 100, 'id': '2e371cca3dcfe383a5058c4410920c1e', 'stats': {}, 'cookie
_mask': 0, 'instructions': [{'instruction_type': 'apply_actions', 'actions': [{'port': 2, 'action_type': 'output'}]}]}, {'switch': '00:00:00:00:00:00:00:01', 'table_id': 0, 'match': {'
in_port': 1, 'dl_vlan': 2}, 'priority': 4000, 'idle_timeout': 0, 'hard_timeout': 0, 'cookie': 100, 'id': '8f4ebd12d31dba67b36dadfad7ba2c84', 'stats': {}, 'cookie_mask': 0, 'instruction
s': [{'instruction_type': 'apply_actions', 'actions': [{'port': 2, 'action_type': 'output'}]}]}]
2023-08-31 16:43:14,872 - INFO [kytos.napps.kytos/flow_manager] (thread_pool_app_12) Send FlowMods batching will sleep for 5 seconds before sending slice[2:2]
kytos $>                                                                                                                                                                                

kytos $> 2023-08-31 16:43:19,878 - INFO [kytos.napps.kytos/flow_manager] (thread_pool_app_12) Sending FlowMods slice[2: 4], iteration: 1, flows[0, 1]: [{'switch': '00:00:00:00:00:00:00
:01', 'table_id': 0, 'match': {'in_port': 1, 'dl_vlan': 3}, 'priority': 4000, 'idle_timeout': 0, 'hard_timeout': 0, 'cookie': 100, 'id': '6f614c8654066e827cce5dc6b3e785cb', 'stats': {}
, 'cookie_mask': 0, 'instructions': [{'instruction_type': 'apply_actions', 'actions': [{'port': 2, 'action_type': 'output'}]}]}]

Sent event with 100 flows with batching, it worked as expected (I truncated the output here):

kytos $> def create_flows(dpid: str, n: int) -> dict[list]: 
    ...:     """docstring.""" 
    ...:     flows = [ 
    ...:         { 
    ...:             "priority": i, 
    ...:             "cookie": i, 
    ...:             "match": {"in_port": 1, "dl_vlan": i}, 
    ...:             "actions": [{"action_type": "output", "port": 2}], 
    ...:         } 
    ...:         for i in range(1, n + 1) 
    ...:     ] 
    ...:     return { 
    ...:         "flow_dict": { 
    ...:             "flows": flows,
    ...:         }, 
    ...:         "dpid": dpid, 
                 "force": True, 
    ...:         "batch_interval": 1, 
    ...:         "batch_size": 50
    ...:     } 
    ...:  
    ...:  
    ...: data = create_flows(dpid, 100) 
    ...: ev = KytosEvent("kytos.flow_manager.flows.install", content=data) 
    ...: controller.buffers.app.put(ev)                                                                                                                                                 

2023-08-31 16:49:04,584 - INFO [kytos.napps.kytos/flow_manager] (thread_pool_app_19) Sending FlowMods slice[0: 50], iteration: 0, flows[0, 50]: ...
2023-08-31 16:49:04,660 - INFO [kytos.napps.kytos/flow_manager] (thread_pool_app_19) Send FlowMods batching will sleep for 1 seconds before sending slice[50:50]
kytos $>                                                                                                                                                                                

kytos $> 2023-08-31 16:49:05,662 - INFO [kytos.napps.kytos/flow_manager] (thread_pool_app_19) Sending FlowMods slice[50: 100], iteration: 1, flows[0, 50]: ...

End-to-End Tests

tests/test_e2e_01_kytos_startup.py ..                                    [  0%]
tests/test_e2e_05_topology.py ..................                         [  8%]
tests/test_e2e_10_mef_eline.py ..........ss.....x.....x................  [ 24%]
tests/test_e2e_11_mef_eline.py ......                                    [ 27%]
tests/test_e2e_12_mef_eline.py .....Xx.                                  [ 30%]
tests/test_e2e_13_mef_eline.py .....xs.s......xs.s.XXxX.xxxx..X......... [ 47%]
...                                                                      [ 48%]
tests/test_e2e_14_mef_eline.py x                                         [ 49%]
tests/test_e2e_15_mef_eline.py ..                                        [ 50%]
tests/test_e2e_20_flow_manager.py .....................                  [ 58%]
tests/test_e2e_21_flow_manager.py ...                                    [ 60%]
tests/test_e2e_22_flow_manager.py ...............                        [ 66%]
tests/test_e2e_23_flow_manager.py ..............                         [ 72%]
tests/test_e2e_30_of_lldp.py ....                                        [ 73%]
tests/test_e2e_31_of_lldp.py ...                                         [ 75%]
tests/test_e2e_32_of_lldp.py ...                                         [ 76%]
tests/test_e2e_40_sdntrace.py .............                              [ 81%]
tests/test_e2e_41_kytos_auth.py ........                                 [ 85%]
tests/test_e2e_50_maintenance.py ........................                [ 95%]
tests/test_e2e_60_of_multi_table.py .....                                [ 97%]
tests/test_e2e_70_kytos_stats.py .......                                 [100%]
=============================== warnings summary ===============================
------------------------------- start/stop times -------------------------------
= 219 passed, 6 skipped, 11 xfailed, 5 xpassed, 867 warnings in 11382.39s (3:09:42) =

kytos-ng / flow_manager

feat: added support for batching FlowMods #172

Summary

Local Tests

End-to-End Tests