search

DelfiSpace / DelfiPQ-PythonTestScripts

Collection of scripts in python to do functional tests
0 stars 0 forks source link

Test fails if split in two files #1

Open StefanoSperetta opened 4 years ago

StefanoSperetta commented 4 years ago

The ResetService test (https://github.com/DelfiSpace/DelfiPQ-PythonTestScripts/blob/UnitTestFramework/Generic/test_ResetService.py) fails consistently if executed after the PingService (https://github.com/DelfiSpace/DelfiPQ-PythonTestScripts/blob/UnitTestFramework/Generic/test_PingService.py). If, instead the two tests are bundled in the same file and executed one after the other (https://github.com/DelfiSpace/DelfiPQ-PythonTestScripts/blob/UnitTestFramework/ADCS/test_ADCS.py).

By looking at the output HTML, it seems the connection to PQ9EGSE is re-established per each test file (despite the fixture is "session") and it also seems the EGSE is not outputting any command on the bus just after the ResetService test starts. After about 5sec a housekeeping request is sent on the bus (and a response is received immediately) but anyway the tests fails (timeout).

@CasperBroekhuizen any idea?

CasperBroekhuizen commented 4 years ago

Weird errors indeed! I had odd behavior as well, but actually different issues when doing: pytest --destination=EPS Generic

the first ping tests would already immediately fail, telling me the bus is unresponsive. However, manually running the functions work fine... I took the liberty of cleaning the pq9client fixture by removing the globals and moving the opening and closing of the port to the conftest.py. As this might affect the scoping, due to the pq9client being placed in a testfile.

For me this fixed the issues when running the tests. The 'Generic' tests now seem to all pass, except the powerCycle (it is not implemented in the EPS i was using)

I've commited the changes, would you mind testing the Generic again? (https://github.com/DelfiSpace/DelfiPQ-PythonTestScripts/commit/576c2759e35312d4fc1920c884c30b0794229646)

StefanoSperetta commented 4 years ago

I just pushed another way of running the tests (d2b2a5b31ba5e9c2242c4031da4dca293c9acd19). All tests are now in the Generic folder (if we stick to this mechanism, we might change again the directory structure).

Now I am enumerating all test_*.py files and selecting in the conftest.py which "system" to test based on a dictionary (PQ9TestSuite.py). Tests can be run from the Generic folder using

pytest --system=ADCS

to run the test suite (running all ADCS-specific tests) or simply

pytest test_PingService.py --destination=ADCS

to run a single test specifying the destination. What do you think?