search

amyreese / fissix

backport of lib2to3, with enhancements
Other
45 stars 22 forks source link

fissix.tests (unittest.loader._FailedTest) ... FAIL #20

Closed sten0 closed 4 years ago

sten0 commented 4 years ago

Description

fissix.tests (unittest.loader._FailedTest) fail. This one stumped me and I had to consult a colleague. P.S. sorry it's taking so long to get Bowler into Debian!

python3.8 -m unittest discover -v 
fissix.tests (unittest.loader._FailedTest) ... FAIL

======================================================================
FAIL: fissix.tests (unittest.loader._FailedTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_fissix/build/fissix/tests/__init__.py", line 9, in load_tests
    return load_package_tests(os.path.dirname(__file__), *args)
  File "/usr/lib/python3.8/test/support/__init__.py", line 243, in load_package_tests
    package_tests = loader.discover(start_dir=pkg_dir,
AssertionError: Path must be within the project

----------------------------------------------------------------------
Ran 1 test in 0.000s

My colleague writes that this test fails due to a cause related to https://github.com/python/cpython/blob/master/Lib/test/support/__init__.py#L138-L140 and assumptions about a specific cpython layout. This sounds like an actionable bug with the following two resolution cases:

1) the system cpython loader should be executed. 2) fissix should included a forked version of the loader or somehow advise/hint the loader.

Details

graingert commented 4 years ago

@sten0 can you run the tests with make tests instead? https://github.com/jreese/fissix/blob/master/makefile#L35

sten0 commented 4 years ago

Thomas Grainger notifications@github.com writes:

@sten0 can you run the tests with make tests instead? https://github.com/jreese/fissix/blob/master/makefile#L35

Done! In both cases manually running

from the source dir:

python3.8 -m pytest --verbose tests fissix/tests/                        
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.8.1, pluggy-0.13.0 -- /usr/bin/python3.8
cachedir: .pytest_cache
rootdir: /build/fissix-eGdcOl/fissix-20.5.1
collected 0 items                                                              

========================= no tests ran in 0.00 seconds =========================
ERROR: file not found: tests

from the build dir (srcdir/.pybuild/cpython3_3.8_fissix/build):

python3.8 -m pytest --verbose tests fissix/tests/              
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.8.1, pluggy-0.13.0 -- /usr/bin/python3.8
cachedir: .pytest_cache
rootdir: /build/fissix-eGdcOl/fissix-20.5.1
collected 0 items                                                              

========================= no tests ran in 0.00 seconds =========================
ERROR: file not found: tests

These methods will not be compatible with our CI (the next step after successful build, tests, and packaging) which runs tests against the packages as-installed-from-debs on a live (but isolated) system.

I've also tried using our integrated PYTEST method (export PYBUILD_TEST_PYTEST=1 in debian/rules) to no avail, other than a more useful backtrace.

   dh_auto_test -O--buildsystem=pybuild
D: pybuild pybuild:560: version: 4.20200804
D: pybuild pybuild:561: ['/usr/bin/pybuild', '--test', '-i', 'python{version}', '-p', '3.8']
D: pybuild pybuild:36: cfg: Namespace(after_build=None, after_clean=None, after_configure=None, after_install=None, after_test=None, before_build=None, before_clean=None, before_configure=None, before_install=None, before_test=None, build_args=None, build_only=False, clean_args=None, clean_only=False, configure_args=None, configure_only=False, custom_tests=True, destdir='debian/tmp', detect_only=False, dir='/<<PKGBUILDDIR>>', disable=None, ext_destdir=None, ext_pattern='\\.so(\\.[^/]*)?$', ext_sub_pattern=None, ext_sub_repl=None, install_args=None, install_dir=None, install_only=False, interpreter=['python{version}'], list_systems=False, name='fissix', print_args=None, quiet=False, really_quiet=False, system=None, test_args=None, test_nose=False, test_nose2=False, test_only=True, test_pytest=True, test_tox=False, verbose=True, versions=['3.8'])
D: pybuild tools:232: invoking: /usr/bin/dpkg-architecture
D: pybuild pybuild:131: detected build system: distutils (certainty: 61%)
I: pybuild base:217: cd /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_fissix/build; python3.8 -m pytest                                                                
D: pybuild tools:232: invoking: cd /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_fissix/build; python3.8 -m pytest 
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.11, py-1.8.1, pluggy-0.13.0
rootdir: /<<PKGBUILDDIR>>
collected 663 items

fissix/tests/test_all_fixers.py .                                        [  0%]
fissix/tests/test_fixers.py ............................................ [  6%]
........................................................................ [ 17%]
........................................................................ [ 28%]
........................................................................ [ 39%]
........................................................................ [ 50%]
........................................................................ [ 61%]
............................................................             [ 70%]
fissix/tests/test_main.py FFF.                                           [ 70%]
fissix/tests/test_parser.py ..F......................................... [ 77%]
......................................................                   [ 85%]
fissix/tests/test_pytree.py ....................................         [ 90%]
fissix/tests/test_refactor.py ....................                       [ 93%]
fissix/tests/test_util.py ........................................       [100%]

=================================== FAILURES ===================================
_____________ TestMain.test_filename_changing_on_output_single_dir _____________

self = <fissix.tests.test_main.TestMain testMethod=test_filename_changing_on_output_single_dir>

    def test_filename_changing_on_output_single_dir(self):
        """2to3 a single directory with a new output dir and suffix."""
        self.setup_test_source_trees()
        out = io.StringIO()
        err = io.StringIO()
        suffix = "TEST"
        ret = self.run_2to3_capture(
            [
                "-n",
                "--add-suffix",
                suffix,
                "--write-unchanged-files",
                "--no-diffs",
                "--output-dir",
                self.py3_dest_dir,
                self.py2_src_dir,
            ],
            io.StringIO(""),
            out,
            err,
        )
        self.assertEqual(ret, 0)
        stderr = err.getvalue()
        self.assertIn(" implies -w.", stderr)
>       self.assertIn(
            "Output in %r will mirror the input directory %r layout"
            % (self.py3_dest_dir, self.py2_src_dir),
            stderr,
        )
E       AssertionError: "Output in '/tmp/tmp37kv_dqo/python3_project' will mirror the input directory '/tmp/tmp37kv_dqo/python2_project' layout" not found in 'WARNING: --write-unchanged-files/-W implies -w.\n'

fissix/tests/test_main.py:100: AssertionError
____________ TestMain.test_filename_changing_on_output_single_file _____________

self = <fissix.tests.test_main.TestMain testMethod=test_filename_changing_on_output_single_file>

    def test_filename_changing_on_output_single_file(self):
        """2to3 a single file with a new output dir."""
        self.setup_test_source_trees()
        err = io.StringIO()
        ret = self.run_2to3_capture(
            [
                "-n",
                "-w",
                "--no-diffs",
                "--output-dir",
                self.py3_dest_dir,
                self.trivial_py2_file,
            ],
            io.StringIO(""),
            io.StringIO(),
            err,
        )
        self.assertEqual(ret, 0)
        stderr = err.getvalue()
>       self.assertIn(
            "Output in %r will mirror the input directory %r layout"
            % (self.py3_dest_dir, self.py2_src_dir),
            stderr,
        )
E       AssertionError: "Output in '/tmp/tmpdzl1liq8/python3_project' will mirror the input directory '/tmp/tmpdzl1liq8/python2_project' layout" not found in ''

fissix/tests/test_main.py:170: AssertionError
_____________ TestMain.test_filename_changing_on_output_two_files ______________

self = <fissix.tests.test_main.TestMain testMethod=test_filename_changing_on_output_two_files>

    def test_filename_changing_on_output_two_files(self):
        """2to3 two files in one directory with a new output dir."""
        self.setup_test_source_trees()
        err = io.StringIO()
        py2_files = [self.trivial_py2_file, self.init_py2_file]
        expected_files = set(os.path.basename(name) for name in py2_files)
        ret = self.run_2to3_capture(
            [
                "-n",
                "-w",
                "--write-unchanged-files",
                "--no-diffs",
                "--output-dir",
                self.py3_dest_dir,
            ]
            + py2_files,
            io.StringIO(""),
            io.StringIO(),
            err,
        )
        self.assertEqual(ret, 0)
        stderr = err.getvalue()
>       self.assertIn(
            "Output in %r will mirror the input directory %r layout"
            % (self.py3_dest_dir, self.py2_src_dir),
            stderr,
        )
E       AssertionError: "Output in '/tmp/tmpfsumfw1k/python3_project' will mirror the input directory '/tmp/tmpfsumfw1k/python2_project' layout" not found in ''

fissix/tests/test_main.py:144: AssertionError
______________ TestPgen2Caching.test_load_grammar_from_subprocess ______________

self = <fissix.tests.test_parser.TestPgen2Caching testMethod=test_load_grammar_from_subprocess>

    @unittest.skipIf(sys.executable is None, "sys.executable required")
    def test_load_grammar_from_subprocess(self):
        tmpdir = tempfile.mkdtemp()
        tmpsubdir = os.path.join(tmpdir, "subdir")
        try:
            os.mkdir(tmpsubdir)
            grammar_base = os.path.basename(support.grammar_path)
            grammar_copy = os.path.join(tmpdir, grammar_base)
            grammar_sub_copy = os.path.join(tmpsubdir, grammar_base)
            shutil.copy(support.grammar_path, grammar_copy)
            shutil.copy(support.grammar_path, grammar_sub_copy)
            pickle_name = pgen2_driver._generate_pickle_name(grammar_copy)
            pickle_sub_name = pgen2_driver._generate_pickle_name(grammar_sub_copy)
>           self.assertNotEqual(pickle_name, pickle_sub_name)
E           AssertionError: '/<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_fissix/.cache/fissix/20.5.1/Grammar3.9.0a6+.pickle' == '/<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_fissix/.cache/fissix/20.5.1/Grammar3.9.0a6+.pickle'

fissix/tests/test_parser.py:74: AssertionError
==================== 4 failed, 659 passed in 20.05 seconds =====================
E: pybuild pybuild:352: test: plugin distutils failed with: exit code=1: cd /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_fissix/build; python3.8 -m pytest            
Traceback (most recent call last):
  File "/usr/bin/pybuild", line 350, in main
    run(func, i, version, c)
  File "/usr/bin/pybuild", line 301, in run
    result = func(context, args)
  File "/usr/share/dh-python/dhpython/build/base.py", line 267, in wrapped_func
    raise Exception(msg)
Exception: exit code=1: cd /<<PKGBUILDDIR>>/.pybuild/cpython3_3.8_fissix/build; python3.8 -m pytest 
dh_auto_test: error: pybuild --test -i python{version} -p 3.8 returned exit code 13
make: *** [debian/rules:8: binary] Error 25
dpkg-buildpackage: error: debian/rules binary subprocess returned exit
status 2

I hope that helps! :-) Thanks, Nicholas

graingert commented 4 years ago

@sten0 can you repost that on the GitHub web UI, it's broken the formatting

graingert commented 4 years ago

@sten0 ah, the version on pypi is missing the @pytest.mark.xfail decorators

@jreese can you make a new release?

graingert commented 4 years ago

@sten0 those tests are expected to fail, please cherry-pick this commit: https://github.com/jreese/fissix/commit/bb4bf03c8d09266d9b6aad693549f3443815e6ef

amyreese commented 4 years ago

Merged the fixes from @graingert and released fissix v20.8.0 to PyPI.

sten0 commented 3 years ago

P.S. Thank you! (it seems like I forgot to follow-up)