Check log messages in tests

[sfmig]

Is your feature request related to a problem? Please describe. In the tests, we are not always checking the error message is the expected one (see here for example). It may be that (by mistake while implementing the test) the test throws an error, but for a different case than the expected one.

Describe the solution you'd like Maybe we should check the error message is the expected one in the test - see here for example.

Describe alternatives you've considered We may consider using pytest's caplog fixture instead. In the example above, I went for what seemed a simpler approach at the time but happy to reconsider. I think using caplog would allow us to check warning messages too (which we are not checking now).

Additional context \

[niksirbi]

Another option is to use the match argument of pytest.raises, i.e. specifically include a regex that's has to match the error message. For example:

with pytest.raises(ValueError, match=r".* 123 .*")

We already use such a construct in some tests, we just don't do it consistently throughout the test suite.

Note that to check the log, we'd need to still use caplog. but testing both the error message and the log each time seems like an overkill (given that we already test the logger separately).