Converted from SourceForge issue 3513638, submitted by ceball
Submit Date: 2012-03-31 12:58 GMT
We have a custom test framework. For instance, topo/tests/init.py has various functions for finding, importing, and running unit tests. Or topo/tests/test_script.py has code for functional tests. There's also topo/tests/test_map_measurement.py. Some of this is documented in topo/tests/README.txt, and some in topo/tests/init.py. Adding new unit tests must be done in such a particular, complex way that nobody does it; adding new functional tests is so simple that anyone can do it, and therefore everyone does it differently (or the same person has done it differently every time...).
Also, there are also various improvements we would like. For instance, maptests stops at the first failure. It would be better if all tests were run, and the number of errors collected. That should happen throughout the functional tests. (There are surely other deficiencies; please add them to this tracker item as you encounter them.)
Meanwhile, there are several mature testing frameworks for Python. A popular one is nose. Additionally, numpy (which also has complex testing requirements) has produced some of their own code for handling test running, and that code integrates with nose. We need to investigate the options to see if we can replace our custom framework with a standard one, which will make it easier to add new tests and to understand existing ones, and should also hopefully address many of the current framework's deficiencies.
Converted from SourceForge issue 3513638, submitted by ceball Submit Date: 2012-03-31 12:58 GMT
We have a custom test framework. For instance, topo/tests/init.py has various functions for finding, importing, and running unit tests. Or topo/tests/test_script.py has code for functional tests. There's also topo/tests/test_map_measurement.py. Some of this is documented in topo/tests/README.txt, and some in topo/tests/init.py. Adding new unit tests must be done in such a particular, complex way that nobody does it; adding new functional tests is so simple that anyone can do it, and therefore everyone does it differently (or the same person has done it differently every time...).
Also, there are also various improvements we would like. For instance, maptests stops at the first failure. It would be better if all tests were run, and the number of errors collected. That should happen throughout the functional tests. (There are surely other deficiencies; please add them to this tracker item as you encounter them.)
Meanwhile, there are several mature testing frameworks for Python. A popular one is nose. Additionally, numpy (which also has complex testing requirements) has produced some of their own code for handling test running, and that code integrates with nose. We need to investigate the options to see if we can replace our custom framework with a standard one, which will make it easier to add new tests and to understand existing ones, and should also hopefully address many of the current framework's deficiencies.