One test failing in test_utils.py when running pytest tests

[mnjowe]

When running pytest tests in the local environment, test_utils fails with the following error:
FAILED tests/test_utils.py::test_logs_parsing - AssertionError: assert 'tlo.methods.demography' in {}

Interestingly, running pytest tests/test_utils.py alone works perfectly fine, and all tests pass. After a quick debug, it appears that the issue is caused by another test, test_logging.py. Specifically, when running pytest tests/test_logging.py tests/test_utils.py, the test_utils fails with the above error. I guess test_logging is modifying some global state, as running the tests in a different order (pytest tests/test_utils.py tests/test_logging.py) resolves the issue, and all tests pass successfully.

@tamuri or @matt-graham can you help looking into this?

[matt-graham]

Hi @mnjowe. Is this running on current master (49acef5). Running

tox -e py311 -- pytest tests/test_utils.py::test_logs_parsing

locally or

pytest tests/test_utils.py::test_logs_parsing

both pass for me without any failures (as does just running pytest tests/test_utils.py).

EDIT: Ignore the above I misread your comment, hadn't realised this was appearing after running in conjuction with tests_logging.py

[matt-graham]

So I think this is happening because of the initialise_logging fixture which is (auto)used in the tests in tests/test_logging.py

https://github.com/UCL/TLOmodel/blob/49acef5288ef0c08c49b806472349ebcaacba8aa/tests/test_logging.py#L69-L83

This was intended to allow testing functions which alter the global logging state without coupling them together by resetting the global state to what is was at the (first) import of tlo.logging.core.

What I didn't account for in this is that the global logging state is affected by the various logger = logging.getLogger(__name__) and related statements in each of the tlo submodules which are executed when they are first imported, which happens after the first import of tlo.logging.core. This means this fixture is removing the side effects on the global logging state of importing these modules. Just removing the line

https://github.com/UCL/TLOmodel/blob/49acef5288ef0c08c49b806472349ebcaacba8aa/tests/test_logging.py#L83

so that the global state is not reset appears to resolve the test failure issue. Ideally it would be good to still allow explicitly resetting the global logging state to a previous value in tests, but I think a context manager solution for doing this might work better - that is the context manager records initial global state, yields to code in a with block which potentially alters global state, and then finalises by restoring global state to initial value. I will try get a PR opened to fix this today!

[mnjowe]

Thanks Matt