Ktmi commented 1 year ago

This PR makes it so that maintenance will now use service interruptions to inform other NApps of which devices are being put into maintenance. Additionally this PR marks all devices in maintenance as being in maintenance through status_reason. This PR resolves the following issues:

76
- Added new status_reason_funcs for Switches, Links, and Interfaces
58
- Added new status_funcs for Switches, Links, and Interfaces
71
44
- ID is now auto generated, rather than set by users
25
- All items are treated correctly
65
7
- All items are validated for existence upon maintenance window creation

Local Test Results

Local tests were performed with kytos-ng/topology#147. With this patch, maintenance windows continued to properly dispatch link up and link down events when maintenance windows began and ended.

End-To-End Tests

Initially end to end tests had an error involving attempting to create a maintenance window with a non-existent switch. This issue is fixed by kytos-ng/kytos-end-to-end-tests#245. When running with the patched end to end tests, the following results were produced:

kytos-end-to-end-tests-kytos-1  | Starting enhanced syslogd: rsyslogd.
kytos-end-to-end-tests-kytos-1  | /etc/openvswitch/conf.db does not exist ... (warning).
kytos-end-to-end-tests-kytos-1  | Creating empty database /etc/openvswitch/conf.db.
kytos-end-to-end-tests-kytos-1  | Starting ovsdb-server.
kytos-end-to-end-tests-kytos-1  | Configuring Open vSwitch system IDs.
kytos-end-to-end-tests-kytos-1  | Starting ovs-vswitchd.
kytos-end-to-end-tests-kytos-1  | Enabling remote OVSDB managers.
kytos-end-to-end-tests-kytos-1  | + sed -i 's/STATS_INTERVAL = 60/STATS_INTERVAL = 7/g' /var/lib/kytos/napps/kytos/of_core/settings.py
kytos-end-to-end-tests-kytos-1  | + sed -i 's/CONSISTENCY_MIN_VERDICT_INTERVAL =.*/CONSISTENCY_MIN_VERDICT_INTERVAL = 60/g' /var/lib/kytos/napps/kytos/flow_manager/settings.py
kytos-end-to-end-tests-kytos-1  | + sed -i 's/LINK_UP_TIMER = 10/LINK_UP_TIMER = 1/g' /var/lib/kytos/napps/kytos/topology/settings.py
kytos-end-to-end-tests-kytos-1  | + sed -i 's/DEPLOY_EVCS_INTERVAL = 60/DEPLOY_EVCS_INTERVAL = 5/g' /var/lib/kytos/napps/kytos/mef_eline/settings.py
kytos-end-to-end-tests-kytos-1  | + sed -i 's/LLDP_LOOP_ACTIONS = \["log"\]/LLDP_LOOP_ACTIONS = \["disable","log"\]/' /var/lib/kytos/napps/kytos/of_lldp/settings.py
kytos-end-to-end-tests-kytos-1  | + sed -i 's/LLDP_IGNORED_LOOPS = {}/LLDP_IGNORED_LOOPS = {"00:00:00:00:00:00:00:01": \[\[4, 5\]\]}/' /var/lib/kytos/napps/kytos/of_lldp/settings.py
kytos-end-to-end-tests-kytos-1  | + sed -i 's/CONSISTENCY_COOKIE_IGNORED_RANGE =.*/CONSISTENCY_COOKIE_IGNORED_RANGE = [(0xdd00000000000000, 0xdd00000000000009)]/g' /var/lib/kytos/napps/kytos/flow_manager/settings.py
kytos-end-to-end-tests-kytos-1  | + sed -i 's/LIVENESS_DEAD_MULTIPLIER =.*/LIVENESS_DEAD_MULTIPLIER = 3/g' /var/lib/kytos/napps/kytos/of_lldp/settings.py
kytos-end-to-end-tests-kytos-1  | + kytosd --help
kytos-end-to-end-tests-kytos-1  | + sed -i s/WARNING/INFO/g /etc/kytos/logging.ini
kytos-end-to-end-tests-kytos-1  | + test -z ''
kytos-end-to-end-tests-kytos-1  | + TESTS=tests/
kytos-end-to-end-tests-kytos-1  | + test -z ''
kytos-end-to-end-tests-kytos-1  | + RERUNS=2
kytos-end-to-end-tests-kytos-1  | + python3 scripts/wait_for_mongo.py
kytos-end-to-end-tests-kytos-1  | Trying to run hello command on MongoDB...
kytos-end-to-end-tests-kytos-1  | Trying to run 'hello' command on MongoDB...
kytos-end-to-end-tests-kytos-1  | Ran 'hello' command on MongoDB successfully. It's ready!
kytos-end-to-end-tests-kytos-1  | + python3 -m pytest tests/test_e2e_50_maintenance.py
kytos-end-to-end-tests-kytos-1  | ============================= test session starts ==============================
kytos-end-to-end-tests-kytos-1  | platform linux -- Python 3.9.2, pytest-7.2.0, pluggy-1.2.0
kytos-end-to-end-tests-kytos-1  | rootdir: /tests
kytos-end-to-end-tests-kytos-1  | plugins: rerunfailures-10.2, timeout-2.1.0, anyio-3.6.2
kytos-end-to-end-tests-kytos-1  | collected 24 items
kytos-end-to-end-tests-kytos-1  | 
kytos-end-to-end-tests-kytos-1  | tests/test_e2e_50_maintenance.py ........................                [100%]
kytos-end-to-end-tests-kytos-1  | 
kytos-end-to-end-tests-kytos-1  | =============================== warnings summary ===============================
kytos-end-to-end-tests-kytos-1  | test_e2e_50_maintenance.py: 17 warnings
kytos-end-to-end-tests-kytos-1  |   /usr/lib/python3/dist-packages/mininet/node.py:1121: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
kytos-end-to-end-tests-kytos-1  |     return ( StrictVersion( cls.OVSVersion ) <
kytos-end-to-end-tests-kytos-1  | 
kytos-end-to-end-tests-kytos-1  | test_e2e_50_maintenance.py: 17 warnings
kytos-end-to-end-tests-kytos-1  |   /usr/lib/python3/dist-packages/mininet/node.py:1122: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
kytos-end-to-end-tests-kytos-1  |     StrictVersion( '1.10' ) )
kytos-end-to-end-tests-kytos-1  | 
kytos-end-to-end-tests-kytos-1  | -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
kytos-end-to-end-tests-kytos-1  | ------------------------------- start/stop times -------------------------------
kytos-end-to-end-tests-kytos-1  | ================= 24 passed, 34 warnings in 1574.08s (0:26:14) =================

viniarck commented 1 year ago

David, when you need a final review let me know, I closed some of the threads that you addressed, I can do another pass later, but since you're pushing code and still have discussion threads open here, I'll wait for you confirmation first.

Good job. It's great how it's shaping up.

viniarck commented 1 year ago

David, when you need a final review let me know, I closed some of the threads that you addressed, I can do another pass later, but since you're pushing code and still have discussion threads open here, I'll wait for you confirmation first.

Good job. It's great how it's shaping up.

Also, if you could follow our Kytos-ng template PR that'd be great, facilitating for everyone. Your PRs descriptions are great, but in the template we also have information about local tests and e2e, which are very helpful for reviewers.

viniarck commented 1 year ago

David, when also exploring locally here, I noticed that finished MWs are also loaded again in the scheduler, probably won't impact at all, but do we need to load them again? From what I can tell, the only utility for a finished mw is to be listed again, which will be directly from the DB. Let me know if you had any other reason

2023-06-23 15:16:28,244 - INFO [kytos.napps.kytos/maintenance] [scheduler.py:157:_schedule] (MainThread) Scheduling "some_mw1"
2023-06-23 15:16:28,244 - INFO [kytos.napps.kytos/maintenance] [scheduler.py:157:_schedule] (MainThread) Scheduling "some_mw2"
2023-06-23 15:16:28,257 - INFO [kytos.napps.kytos/maintenance] [scheduler.py:173:_schedule] (MainThread) Scheduled "some_mw2" end at 2023-06-23 18:20:00+00:00
2023-06-23 15:16:28,257 - INFO [apscheduler.scheduler] [base.py:445:add_job] (MainThread) Adding job tentatively -- it will be properly scheduled when the scheduler starts
2023-06-23 15:16:28,257 - INFO [kytos.napps.kytos/maintenance] [scheduler.py:157:_schedule] (MainThread) Scheduling "some_mw3"
2023-06-23 15:16:28,257 - INFO [apscheduler.scheduler] [base.py:445:add_job] (MainThread) Adding job tentatively -- it will be properly scheduled when the scheduler starts

Other than that, I explored a link and and a sw under maintenance, they behaved as expected, I also simulated restarting controller when a MW hasn't been started yet, and it picked up and loaded correctly too. Once your local tests are also positive and e2e tests are passing this one can land too, also if Italo hasn't started exploratory tests yet, we can potentially merge this if you wish to take the opportunity, otherwise it'll be on 2023.1.1, let me know what which way you prefer.

Ktmi commented 1 year ago

I was able to do end to end tests with both this and kytos-ng/topology#147 together, and most of the tests passed, except 3 of them which used a non-existant switch id. The switch ID issue will be fixed with kytos-ng/kytos-end-to-end-tests#245, which replaces those switch ids with valid existing ones. Doing the end to end tests with the fixed switch ids results in all maintenance tests passing.

Ktmi commented 1 year ago

David, when also exploring locally here, I noticed that finished MWs are also loaded again in the scheduler, probably won't impact at all, but do we need to load them again? From what I can tell, the only utility for a finished mw is to be listed again, which will be directly from the DB. Let me know if you had any other reason

Loading them again is completely unnecessary. I'll write a method on the DB controller just for loading the unfinished windows only.

viniarck commented 1 year ago

Let's ship it @Ktmi, let me know if you'll merge it or if I should do it.

kytos-ng / maintenance

Add service interruptions to maintenance windows #78

76

58

71

44

25

65

7

Local Test Results

End-To-End Tests