Flows created outside Kytos should be removed by consistency check

[ajoaoff]

If a flow is created outside Kytos, it should be removed by the consistency check, but that is not happening. Steps to reproduce:

• start kytos with a clean storehouse;
• start Mininet with a simple topology (e.g. linear with 3 switches);
• use OVS to install a flow (ovs-ofctl add-flow s1 "<flow>");
• wait for the consistency check to run;
• verify that the flow is still installed.

[cmagnobarbosa]

Additional information reported by Antonio: The inconsistency has been detected (a warning is shown on the console), but the flow has not been removed from the switch.

[cmagnobarbosa]

I can't reproduce this bug, in my tests the outside flow is removed from the switch. If you can please provide more information how to reproduce this bug.

Steps :

• Start kytos

• Create a topology using the mininet sudo mn --topo linear,3 --mac --controller=remote,ip=127.0.0.1 --switch ovsk

• Enable switches

  for i in 1 2 3; do
  echo "switch: $i"
  curl -X POST "http://127.0.0.1:8181/api/kytos/topology/v3/switches/00:00:00:00:00:00:00:0$i/enable"
  done

• Enable all interfaces:

for i in 1 2 3; do
  echo "switch: $i"
  curl -X POST "http://127.0.0.1:8181/api/kytos/topology/v3/interfaces/switch/00:00:00:00:00:00:00:0$i/enable"
done

• Now, install an "alien" flow:
  • In mininet console sh ovs-ofctl add-flow s1 "table=0, dl_src=01:00:00:00:00:00/01:00:00:00:00:00,action=drop"
• Check if the flow has been added to s1 sh ovs-ofctl dump-flows s1
  • output:

    cookie=0x0, duration=1.533s, table=0, n_packets=0, n_bytes=0, dl_src=01:00:00:00:00:00/01:00:00:00:00:00 actions=drop
    cookie=0x0, duration=31.666s, table=0, n_packets=7, n_bytes=294, priority=1000,dl_vlan=3799,dl_type=0x88cc
    actions=CONTROLLER:65535
    cookie=0x0, duration=31.663s, table=0, n_packets=7, n_bytes=756, priority=0 actions=CONTROLLER:65535

• Wait for the consistency check, when the consistency check is performed, a log message related to the inconsistency is generated

  2021-03-17 09:40:59,472 - INFO [kytos.napps.kytos/flow_manager] (flow_manager) A consistency problem was detected in
  switch 00:00:00:00:00:00:00:01.
  2021-03-17 09:40:59,479 - INFO [kytos.napps.kytos/flow_manager] (flow_manager) Flow forwarded to switch
  00:00:00:00:00:00:00:01 to be deleted.

• Now, check for flows installed in switch s1: sh ovs-ofctl dump-flows s1

  • The "alien" flow has been deleted.

  cookie=0x0, duration=772.606s, table=0, n_packets=248, n_bytes=10416, priority=1000,dl_vlan=3799,dl_type=0x88cc
  actions=CONTROLLER:65535
  cookie=0x0, duration=772.603s, table=0, n_packets=21, n_bytes=2268, priority=0 actions=CONTROLLER:65535
  

  When the consistency check is performed the inconsistency warning is not shown:

• in console:

      2021-03-17 10:32:59,830 - INFO [kytos.napps.kytos/flow_manager] (flow_manager) Flows loaded.
      2021-03-17 10:33:59,836 - INFO [kytos.napps.kytos/flow_manager] (flow_manager) Flows loaded.

• The inconsistency has solved.

Additional information:

• Checking with wireshark the OFPT_FLOW_MOD with delete command to remove the "alien" flow was sent normally.

[hdiogenes]

@ajoaoff is this issue the same that generated this error in the end-to-end tests?

self = <tests.test_e2e_20_flow_manager.TestE2EFlowManager instance at 0x7fbbe40f3368>
restart_kytos = True

   def modify_match(self, restart_kytos=False):
       """Test if after a match is modified outside kytos, the original
          flow is restored."""
       self.net.restart_kytos_clean()
       time.sleep(5)

       payload = {
           "flows": [
               {
               "priority": 10,
               "idle_timeout": 360,
               "hard_timeout": 1200,
               "match": {
                   "in_port": 1
               },
               "actions": [
                   {
                   "action_type": "output",
                   "port": 2
                   }
               ]
               }
           ]
       }

       api_url = KYTOS_API + '/flow_manager/v2/flows/00:00:00:00:00:00:00:01'
       response = requests.post(api_url, data=json.dumps(payload),
                                headers={'Content-type': 'application/json'})
       assert response.status_code == 200
       data = response.json()
       assert 'FlowMod Messages Sent' in data['response']

       # wait for the flow to be installed
       time.sleep(20)

       s1 = self.net.net.get('s1')
       s1.dpctl('del-flows', 'in_port=1')
       s1.dpctl('add-flow', 'idle_timeout=360,hard_timeout=1200,priority=10,'
                'dl_vlan=324,actions=output:1')
       if restart_kytos:
           # restart controller keeping configuration
           self.net.start_controller()

       time.sleep(60)

       s1 = self.net.net.get('s1')
       flows_s1 = s1.dpctl('dump-flows')
     assert len(flows_s1.split('\r\n ')) == 2
E       assert 3 == 2
E        +  where 3 = len([' cookie=0x0, duration=58.458s, table=0, n_packets=16, n_bytes=672, priority=1000,dl_vlan=3799,dl_type=0x88cc actions...packets=0, n_bytes=0, idle_timeout=360, hard_timeout=1200, priority=10,in_port="s1-eth1" actions=output:"s1-eth2"\r\n'])
E        +    where [' cookie=0x0, duration=58.458s, table=0, n_packets=16, n_bytes=672, priority=1000,dl_vlan=3799,dl_type=0x88cc actions...packets=0, n_bytes=0, idle_timeout=360, hard_timeout=1200, priority=10,in_port="s1-eth1" actions=output:"s1-eth2"\r\n'] = <built-in method split of str object at 0x562b18800270>('\r\n ')
E        +      where <built-in method split of str object at 0x562b18800270> = ' cookie=0x0, duration=58.458s, table=0, n_packets=16, n_bytes=672, priority=1000,dl_vlan=3799,dl_type=0x88cc actions=..._packets=0, n_bytes=0, idle_timeout=360, hard_timeout=1200, priority=10,in_port="s1-eth1" actions=output:"s1-eth2"\r\n'.split

tests/test_e2e_20_flow_manager.py:276: AssertionError

[ajoaoff]

Yes, @hdiogenes, I opened the issue because of that test.

[ajoaoff]

@cmagnobarbosa I teste with the same flow you used and it was really removed by the consistency check, but with the flow I had used, it wasn't. The flow I used was "table=1,priority=5,in_port=2,actions=output:2".

[cmagnobarbosa]

Hello @ajoaoff, in my first tests using the flow table = 1, priority = 5, in_port = 2, actions = output:2, it is also removed by the consistency check. But I'm going to do some other tests.

Command used to add the flow:

• sh ovs-ofctl add-flow s1 "table=1,priority=5,in_port=2,actions=output:2"

kytos bug-report:

Platform

Release information

Distributor ID: Ubuntu Description: Ubuntu 20.10 Release: 20.10 Codename: groovy

System Information

Linux carlosmagno-Lenovo-ideapad-330S-15IKB 5.8.0-25-generic #26-Ubuntu SMP Thu Oct 15 10:30:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Python

path=/home/carlosmagno/workspace/kytos/Teste_Coverage/env_issue_113/bin/python version=Python 3.6.12

Pip

path=/home/carlosmagno/workspace/kytos/Teste_Coverage/env_issue_113/bin/pip version=pip 21.0.1 from /home/carlosmagno/workspace/kytos/Teste_Coverage/env_issue_113/lib/python3.6/site-packages/pip (python 3.6)

Python Packages

kytos | github.com:cmagnobarbosa/kytos | 35e3868d
kytos_flow_manager | github.com:cmagnobarbosa/flow_manager | b916cde2
kytos_mef_eline | github.com:kytos/mef_eline | f066c41f
kytos_of_core | github.com:cmagnobarbosa/of_core | f8e3d0e1
kytos_of_l2ls | github.com:cmagnobarbosa/of_l2ls | b2baf9a2
kytos_of_lldp | github.com:cmagnobarbosa/of_lldp | 37d5bdf0
kytos_pathfinder | github.com/kytos/pathfinder | d8ba617a
kytos_storehouse | github.com:cmagnobarbosa/storehouse | 0c7e4ad5
kytos_topology | github.com:cmagnobarbosa/topology | 53d710bc
kytos_utils | github.com:cmagnobarbosa/kytos-utils | a16a659e
python_openflow | github.com:cmagnobarbosa/python-openflow | 50f29d34

Kytos environment

Kytosd

path=/home/carlosmagno/workspace/kytos/Teste_Coverage/env_issue_113/bin/kytosd version=kytosd 2020.2

Kytos

path=/home/carlosmagno/workspace/kytos/Teste_Coverage/env_issue_113/bin/kytos version=kytos command line, version 2020.2

Installed napps

Status | NApp ID | Description
=======+========================+================================================================================== [ie] | kytos/flow_manager:3.0 | Manage switches' flows through a REST API.
[ie] | kytos/mef_eline:2.4 | NApp to provision circuits from user request
[ie] | kytos/of_core:1.5.1 | OpenFlow Core of Kytos Controller, responsible for main OpenFlow operations.
[ie] | kytos/of_l2ls:1.2.0 | A L2 learning switch application for OpenFlow switches.
[ie] | kytos/of_lldp:1.1 | Discover network-to-network interfaces (NNIs) using the LLDP protocol.
[ie] | kytos/pathfinder:2.2.3 | Keeps track of topology changes, and calculates the best path between two points. [ie] | kytos/storehouse:1.3.2 | Persistence NApp with support for multiple backends
[ie] | kytos/topology:3.7.1 | Manage the network topology.

[josemauro]

I did 2 tests. In the first one I did not enabled interfaces and switches but I installed the alien flow. The consistency check did not work. The second test I followed the exactly steps described by @cmagnobarbosa (enabling switches and interfaces) and the consistency check worked, the alien flow was removed. I used this command to install the alien flow:

sh ovs-ofctl add-flow s1 "table=1,priority=5,in_port=2,actions=output:2"

[cmagnobarbosa]

Solving problem #114 can solve that problem. This problem may be related to the way the flows are compared. Also, Italo reported that this problem only occurs after kytos is restarted.