neuronsimulator / nrn

NEURON Simulator
http://nrn.readthedocs.io
Other
412 stars 118 forks source link

Remove stdout/stderr output from Python tests #3115

Open JCGoran opened 2 months ago

JCGoran commented 2 months ago

Running ctest -V -R pytest_coreneuron::basic_tests_py results in almost 1700 lines of output, and this makes it very hard to debug the tests when there's an actual error. We can use pdb for interactive debugging by just inserting a breakpoint() somewhere in the test file (if we know where the error could be), or running python -m pdb -m pytest [args] (if we don't), completely eliminating the need for any print statements.

Note that this applies to all Python-based tests, not just pytest_coreneuron::basic_tests_py.

An alternative would be to use some kind of a flag (env variable maybe?) to toggle enabling output to stdout/stderr in the tests.

1uc commented 1 month ago

I looked at something similar and found that often the pattern:

ERROR: Something terrible happened!

was due to:

    with pytest.raises(Exception):
        do_something_bad()

while doing so we eventually encounter the equivalent of:

printf_callback(stderr, "ERROR: Something ... \n");

, where printf_callback is a function pointer and there's an infrastructure to set the pointer to something else.

In those cases one could try to register a different callback, that just prints to a string (that we then ignore or inspect to make sure it contain the word "ERROR").