testify doesn't report the import error correctly when test path is given as a file path

[baris]

The output is pretty confusing. The root of the issue is test_discovery.py:discover:discover_inner nested function. It only gives the extra information if the second try to import the module is successful.

In the first case below second try was successful since testify tried to import the parent package test.plugins when it wasn't able to import test.plugings.violation_collector_test.

The fix looks easy, but I think the code in discover function needs to be refactored and needs test coverage.

$ testify test/plugins -v
DISCOVERY FAILURE!
There was a problem importing one or more tests:
Failed to find module test/plugins

No tests were discovered (tests must subclass TestCase and test methods must begin with 'test').
ERROR.  0 tests / 0 cases: 0 passed, 0 failed.  (Total test time 0.00s)

$ testify test.plugins -v
DISCOVERY FAILURE!
There was a problem importing one or more tests:
discovery(test.plugins.violation_collector_test) failed: module <module 'test.plugins' from './test/plugins/__init__.pyc'> has no attribute 'violation_collector_test'; this is most likely due to earlier error ImportError('No module named non_existent_module',)

No tests were discovered (tests must subclass TestCase and test methods must begin with 'test').
ERROR.  0 tests / 0 cases: 0 passed, 0 failed.  (Total test time 0.00s)

$ cat test_import_error.py
import testify as T
import non_existent_module

class ImportErrorTest(T.TestCase):
    def test_nothing(self):
        pass

$ testify test_import_error.py -v
DISCOVERY FAILURE!
There was a problem importing one or more tests:
Failed to find module test_import_error.py

No tests were discovered (tests must subclass TestCase and test methods must begin with 'test').
ERROR.  0 tests / 0 cases: 0 passed, 0 failed.  (Total test time 0.00s)

[mrtyler]

Possibly related to #14?

[bukzor]

+1

This is extremely annoying, and also concerning, since this should be that special of a case.

[asottile]

Fixed via #304