search

pierrepo / PBxplore

A suite of tools to explore protein structures with Protein Blocks :snake:
https://pbxplore.readthedocs.org/en/latest/
MIT License
28 stars 17 forks source link

temporary directories in test_tmp directory are not erased after tests #43

Closed pierrepo closed 9 years ago

pierrepo commented 9 years ago

Maybe use a TearDown function? http://nose.readthedocs.org/en/latest/writing_tests.html

jbarnoud commented 9 years ago

It could work. I'll have a look.

Just as a remainder, here is the cause of the issue : most tests call a function that create a temporary directory, call PBassign in this directory, check some assertions, and finally delete the directory. The point is that if a test fails, the temporary directory is still there to investigate the reasons of the failure. Some tests are supposed to fail, though. Thanks to a custom decorator, these tests pass only if they raise an exception. But if the exception is raised, then its execution stops before the point where the temporary directory is deleted.

I found ways to delete the temporary directory in all cases, but none that allows to keep it if the test fails; especially to keep it if a failure test fails (i.e. if the expected excpection is not raised).

I'll check the teardown method to see what I can do with them.

pierrepo commented 9 years ago

OK. Why not erasing the temp directories in all cases? In case of failure, we could return to the user the exact command that failed and the file with expected output. If the user wants to understand further the error, he rerun the command that failed and check by himslef. It would be much simplifier, especially for average users who will not got any further after the failure message.

jbarnoud commented 9 years ago

On 12/03/15 16:33, Pierre Poulain wrote:

OK. Why not erasing the temp directories in all cases? In case of failure, we could return to the user the exact command that failed and the file with expected output.

It will most likely ends like that if I do not find a cleaner way. I do not like it, though. It adds steps in the debug process when debugging is already painful enough.

If the user wants to understand further the error, he rerun the command that failed and check by himslef. It would be much simplifier, especially for average users who will not got any further after the failure message.

These tests are not really for the average user. They mostly are for us, developers, to make sure we are not breaking anything. I would expect the average user to run the demos instead.

— Reply to this email directly or view it on GitHub https://github.com/pierrepo/PBxplore/issues/43#issuecomment-78504098.

pierrepo commented 9 years ago

+1 for

I would expect the average user to run the demos instead.

So if regression tests are mostly for Travis and developpers, why not cleaning the test_tmp dir at the beginning of the serie of tests?

jbarnoud commented 9 years ago

On 12/03/15 17:16, Pierre Poulain wrote:

+1 for

I would expect the average user to run the demos instead.

So if regression tests are mostly for Travis and developpers, why not cleaning the test_tmp dir at the beginning of the serie of tests?

— Reply to this email directly or view it on GitHub https://github.com/pierrepo/PBxplore/issues/43#issuecomment-78513828.

Travis do not care about the files being cleared or not as a new virtual machine is created at each run of the test suite. For developers however, being able to keep the output of failing tests is convenient for diagnostic. But, obviously, it is only my workflow to compare these output throughout commits. So if I do not manage to have them cleared as they should when all tests pass as they should, then I will have the files cleared in all cases. In this last scenario I may try to have an option to keep the files, but I do not like the idea of having options in the tests.

pierrepo commented 9 years ago

Agreed. But if you clean all temp directories in test_tmp/ when you load test_regression.py, you will start with a fresh and clean environnent.