NCAS-CMS / cf-python

A CF-compliant Earth Science data analysis library
http://ncas-cms.github.io/cf-python
MIT License
125 stars 19 forks source link

Tests: check that intended exception messages emerge #733

Open sadielbartholomew opened 8 months ago

sadielbartholomew commented 8 months ago

To help to avoid bugs whereby an untrue and/or misleading exception message is raised due to some logical issue, such as #732, we should try to test not only that an exception is raised for some cf-python code but that it is the specific one that should be (e.g. ValueError and not just a general Exception) and that the specific error message is correct (bar details that will change which can be selectively ignored), via providing the appropriate string or regex to assertRaisesRegex.

There are (git grep "assertRaises" | wc -l =) 245 uses of assertRaises in the full test suite to test that some exception is raised when it should be, and I imagine most if not all of these can be upgraded in this sense.

See https://github.com/NCAS-CMS/cf-python/pull/732#issuecomment-1977312354 for a small example I have included.