search

sonata-nfv / son-tests

SONATA's Service Platform Tests for Integration and Qualification environments
http://www.sonata-nfv.eu/
Apache License 2.0
0 stars 11 forks source link

Fix the status of the plugin manager at the end of integration of int-4 #290

Open tsoenen opened 7 years ago

tsoenen commented 7 years ago

If Integration test 5 is run directly after integration test 4, the start state of the platform for integration test 5 is corrupted. There are no plugins registered with the plugin manager, while at least the SMR, SLM and IA should be registered. This causes integration test 5 to fail.

If we run integration test 5 on a clean environment, the test succeeds. As temporarily fix, we changed the chain of integration tests, so that integration test 5 is run on a clean environment.

In the long run, we should fix the issue itself. It should not be possible that the platform reaches the corrupted state it has at the end of integration test 4 (this state is not necessarily caused by int-4, it can be caused by an earlier test as well). Therefor, we should investigate how we reached this state and prevent it from happening again.

Additional suggestion by Jose: Add to the test script, so that the test starts from a known state of the platform, no matter what the previous tests did to the platform. This should be done in addition to investigating/fixing the issue described above.

jbonnet commented 7 years ago

@tsoenen As a rule, the order of running the tests should be irrelevant (there are even some testing frameworks that force this). To facilitate/achieve this, there's usual a set-up action, ran before each test, that puts the component to be tested in a know state.

Can't we do it this way?

tsoenen commented 7 years ago

@jbonnet

Yes, that is basically what this quick fix does. It is not as clean as your suggestion (where the cleaning of the platform is done at the start of the test, instead of running int-9 to provide a clean platform), but it fixes the test for now.

Next to making sure that the test starts from a known state, we should also address the real issue that causes this failure: it is somehow possible that our platform reaches a state where the plugins are no longer registered in the plugin manager. Whether this is caused by some standard actions that a owner/customer could do, or by some 'sonata-team admin' moves during the previous test should be checked. But if its the first, then we need to prevent this from happening. This issue shoudl remind us to fix this when we have more time (as I don't think this has the highest priority)

I added the suggestion to the original post.