2.0.6: pytest is failing in two units

[kloczek]

Looks like still it is some issue

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-argcomplete-2.0.6-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-argcomplete-2.0.6-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.8.16, pytest-7.2.2, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/argcomplete-2.0.6, configfile: pyproject.toml, testpaths: test/test.py
plugins: remotedata-0.4.0
collected 148 items

test/test.py ...................................usage: pytest [-h] --foo FOO {bar}
pytest: error: too few arguments
usage: pytest [-h] --foo FOO {bar}
pytest: error: argument --foo is required
...usage: pytest foo [-h] {bar}
pytest foo: error: too few arguments
..x..............x...x.......................x...x..................F.x..xx..x........x.F....xs.......s.......

========================================================================================= FAILURES ==========================================================================================
_________________________________________________________________________________ TestTcsh.test_comp_point __________________________________________________________________________________

self = <test.test.TestTcsh testMethod=test_comp_point>

    def test_comp_point(self):
        # Use environment variable to change how prog behaves
>       self.assertEqual(self.sh.run_command("export POINT=1"), "")
E       AssertionError: 'export: Command not found.\r\n' != ''
E       - export: Command not found.

test/test.py:1185: AssertionError
_________________________________________________________________________________ TestFish.test_comp_point __________________________________________________________________________________

self = <test.test.TestFish testMethod=test_comp_point>

    def test_comp_point(self):
        # Use environment variable to change how prog behaves
        self.assertEqual(self.sh.run_command("export POINT=1"), "")
>       self.assertEqual(self.sh.run_command("prog point hi\t"), "13\r\n")
E       AssertionError: 'hi\r\n' != '13\r\n'
E       - hi
E       + 13

test/test.py:1186: AssertionError
================================================================================== short test summary info ==================================================================================
SKIPPED [2] test/test.py:1091: skip
XFAIL test/test.py::TestSplitLine::test_escaped_special_in_double_quotes - reason:
XFAIL test/test.py::TestBash::test_exclamation_in_double_quotes - reason:
XFAIL test/test.py::TestBash::test_parse_special_characters_dollar - reason:
XFAIL test/test.py::TestBashGlobal::test_exclamation_in_double_quotes - reason:
XFAIL test/test.py::TestBashGlobal::test_parse_special_characters_dollar - reason:
XFAIL test/test.py::TestTcsh::test_continuation - reason:
XFAIL test/test.py::TestTcsh::test_parse_special_characters - reason:
XFAIL test/test.py::TestTcsh::test_parse_special_characters_dollar - reason:
XFAIL test/test.py::TestTcsh::test_quoted_space - reason:
XFAIL test/test.py::TestTcsh::test_unquoted_space - reason:
XFAIL test/test.py::TestFish::test_parse_special_characters - reason:
FAILED test/test.py::TestTcsh::test_comp_point - AssertionError: 'export: Command not found.\r\n' != ''
FAILED test/test.py::TestFish::test_comp_point - AssertionError: 'hi\r\n' != '13\r\n'
============================================================= 2 failed, 133 passed, 2 skipped, 11 xfailed in 124.35s (0:02:04) ==============================================================

[kloczek]

Just in case fish and tcsh are installed in build env.

[kislyuk]

Hello, pytest is not supported by this project (tests are expected to be run using unittest).

[kloczek]

Please be aware of fact that corectly written unittest test suite is corrctly handled by pytest.

[kislyuk]

As you saw in this case, that's incorrect. Pytest does not respect the expected failures markers that were in that test suite.

It's a moot point now, because the failing tests were part of a suite that supported a feature that we will no longer support going forward.

[kloczek]

FYI: now in 3.0.3 pytest is failing completly different way

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-argcomplete-3.0.3-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-argcomplete-3.0.3-2.fc35.x86
_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network'
============================= test session starts ==============================
platform linux -- Python 3.8.16, pytest-7.2.2, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/argcomplete-3.0.3
collected 60 items / 1 error

==================================== ERRORS ====================================
______________ ERROR collecting test/test_package/test_module.py _______________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/argcomplete-3.0.3/test/test_package/test_module.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
test/test_package/test_module.py:3: in <module>
    from test_package import main
E   ModuleNotFoundError: No module named 'test_package'
=========================== short test summary info ============================
ERROR test/test_package/test_module.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.13s ===============================

[kloczek]

3.0.5 is failing in new way Here is pytest output:

```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-argcomplete-3.0.5-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-argcomplete-3.0.5-2.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra -m 'not network' ==================================================================================== test session starts ==================================================================================== platform linux -- Python 3.8.16, pytest-7.2.2, pluggy-1.0.0 benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000) rootdir: /home/tkloczko/rpmbuild/BUILD/argcomplete-3.0.5 plugins: hypothesis-6.70.1, benchmark-4.0.0, mock-3.10.0, pylama-8.4.1, asyncio-0.21.0, flaky-3.7.0 asyncio: mode=strict collected 60 items test/test_contrib_shells.py FFFFFFFFFFFFFFFFFFFFssssssssssssssssssssssssssssssssssssssss [100%] ========================================================================================= FAILURES ========================================================================================== ___________________________________________________________________________ TestShellBase.test_simple_completion ____________________________________________________________________________ self = def test_simple_completion(self): > self.assertEqual(self.sh.run_command("prog basic f\t"), "foo\r\n") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1118: AttributeError ___________________________________________________________________________ TestShellBase.test_partial_completion ___________________________________________________________________________ self = def test_partial_completion(self): > self.assertEqual(self.sh.run_command("prog basic b\tr"), "bar\r\n") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1121: AttributeError ________________________________________________________________________ TestShellBase.test_single_quoted_completion ________________________________________________________________________ self = def test_single_quoted_completion(self): > self.assertEqual(self.sh.run_command("prog basic 'f\t"), "foo\r\n") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1124: AttributeError ________________________________________________________________________ TestShellBase.test_double_quoted_completion ________________________________________________________________________ self = def test_double_quoted_completion(self): > self.assertEqual(self.sh.run_command('prog basic "f\t'), "foo\r\n") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1127: AttributeError _____________________________________________________________________________ TestShellBase.test_unquoted_space _____________________________________________________________________________ self = def test_unquoted_space(self): > self.assertEqual(self.sh.run_command("prog space f\t"), "foo bar\r\n") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1130: AttributeError ______________________________________________________________________________ TestShellBase.test_quoted_space ______________________________________________________________________________ self = def test_quoted_space(self): > self.assertEqual(self.sh.run_command('prog space "f\t'), "foo bar\r\n") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1133: AttributeError ______________________________________________________________________________ TestShellBase.test_continuation ______________________________________________________________________________ self = def test_continuation(self): # This produces 'prog basic foo --', and '--' is ignored. > self.assertEqual(self.sh.run_command("prog basic f\t--"), "foo\r\n") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1137: AttributeError ______________________________________________________________________________ TestShellBase.test_quoted_exact ______________________________________________________________________________ self = def test_quoted_exact(self): > self.assertEqual(self.sh.run_command('prog basic "f\t--'), "foo\r\n") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1144: AttributeError ___________________________________________________________________________ TestShellBase.test_special_characters ___________________________________________________________________________ self = def test_special_characters(self): > self.assertEqual(self.sh.run_command("prog spec d\tf"), "d$e$f\r\n") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1147: AttributeError ____________________________________________________________________ TestShellBase.test_special_characters_single_quoted ____________________________________________________________________ self = def test_special_characters_single_quoted(self): > self.assertEqual(self.sh.run_command("prog spec 'd\tf'"), "d$e$f\r\n") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1152: AttributeError ____________________________________________________________________ TestShellBase.test_special_characters_double_quoted ____________________________________________________________________ self = def test_special_characters_double_quoted(self): > self.assertEqual(self.sh.run_command('prog spec "d\tf"'), "d$e$f\r\n") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1155: AttributeError ________________________________________________________________________ TestShellBase.test_parse_special_characters ________________________________________________________________________ self = def test_parse_special_characters(self): > self.assertEqual(self.sh.run_command("prog spec d$e$\tf"), "d$e$f\r\n") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1158: AttributeError ____________________________________________________________________ TestShellBase.test_parse_special_characters_dollar _____________________________________________________________________ self = def test_parse_special_characters_dollar(self): # First tab expands to 'd\$e\$'; completion works with 'd$' but not 'd\$'. > self.assertEqual(self.sh.run_command('prog spec "d$e\tf\t'), "d$e$f\r\n") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1164: AttributeError ______________________________________________________________________ TestShellBase.test_exclamation_in_double_quotes ______________________________________________________________________ self = def test_exclamation_in_double_quotes(self): # Exclamation marks cannot be properly escaped within double quotes. # 'a!b' == "a"\!"b" > self.assertEqual(self.sh.run_command('prog spec "x\t'), "x!x\r\n") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1169: AttributeError _________________________________________________________________________________ TestShellBase.test_quotes _________________________________________________________________________________ self = def test_quotes(self): > self.assertEqual(self.sh.run_command("prog quote 1\t"), "1'1\r\n") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1172: AttributeError _____________________________________________________________________ TestShellBase.test_single_quotes_in_double_quotes _____________________________________________________________________ self = def test_single_quotes_in_double_quotes(self): > self.assertEqual(self.sh.run_command('prog quote "1\t'), "1'1\r\n") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1176: AttributeError _____________________________________________________________________ TestShellBase.test_single_quotes_in_single_quotes _____________________________________________________________________ self = def test_single_quotes_in_single_quotes(self): # Single quotes cannot be escaped within single quotes. # "a'b" == 'a'\''b' > self.assertEqual(self.sh.run_command("prog quote '1\t"), "1'1\r\n") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1181: AttributeError ____________________________________________________________________________ TestShellBase.test_wordbreak_chars _____________________________________________________________________________ self = def test_wordbreak_chars(self): > self.assertEqual(self.sh.run_command("prog break a\tc"), "a:b:c\r\n") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1184: AttributeError _________________________________________________________________________ TestShellBase.test_completion_environment _________________________________________________________________________ self = def test_completion_environment(self): > self.assertEqual(self.sh.run_command("prog env o\t"), "ok\r\n") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1193: AttributeError _______________________________________________________________________________ TestShellBase.test_comp_point _______________________________________________________________________________ self = def test_comp_point(self): # Use environment variable to change how prog behaves > self.assertEqual(self.sh.run_command("export POINT=1"), "") E AttributeError: 'TestShellBase' object has no attribute 'assertEqual' test/test.py:1197: AttributeError ================================================================================== short test summary info ================================================================================== SKIPPED [1] test/test.py:1195: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1192: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1135: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1126: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1166: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1157: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1162: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1120: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1143: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1132: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1171: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1117: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1123: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1175: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1178: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1146: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1154: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1151: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1129: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1183: tcsh is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1195: fish is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1192: fish is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1135: fish is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1126: fish is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1166: fish is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1157: fish is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1162: fish is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1120: fish is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1143: fish is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1132: fish is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1171: fish is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1117: fish is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1123: fish is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1175: fish is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1178: fish is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1146: fish is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1154: fish is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1151: fish is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1129: fish is not supported. Enable this test manually if needed. SKIPPED [1] test/test.py:1183: fish is not supported. Enable this test manually if needed. FAILED test/test_contrib_shells.py::TestShellBase::test_simple_completion - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' FAILED test/test_contrib_shells.py::TestShellBase::test_partial_completion - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' FAILED test/test_contrib_shells.py::TestShellBase::test_single_quoted_completion - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' FAILED test/test_contrib_shells.py::TestShellBase::test_double_quoted_completion - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' FAILED test/test_contrib_shells.py::TestShellBase::test_unquoted_space - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' FAILED test/test_contrib_shells.py::TestShellBase::test_quoted_space - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' FAILED test/test_contrib_shells.py::TestShellBase::test_continuation - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' FAILED test/test_contrib_shells.py::TestShellBase::test_quoted_exact - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' FAILED test/test_contrib_shells.py::TestShellBase::test_special_characters - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' FAILED test/test_contrib_shells.py::TestShellBase::test_special_characters_single_quoted - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' FAILED test/test_contrib_shells.py::TestShellBase::test_special_characters_double_quoted - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' FAILED test/test_contrib_shells.py::TestShellBase::test_parse_special_characters - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' FAILED test/test_contrib_shells.py::TestShellBase::test_parse_special_characters_dollar - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' FAILED test/test_contrib_shells.py::TestShellBase::test_exclamation_in_double_quotes - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' FAILED test/test_contrib_shells.py::TestShellBase::test_quotes - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' FAILED test/test_contrib_shells.py::TestShellBase::test_single_quotes_in_double_quotes - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' FAILED test/test_contrib_shells.py::TestShellBase::test_single_quotes_in_single_quotes - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' FAILED test/test_contrib_shells.py::TestShellBase::test_wordbreak_chars - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' FAILED test/test_contrib_shells.py::TestShellBase::test_completion_environment - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' FAILED test/test_contrib_shells.py::TestShellBase::test_comp_point - AttributeError: 'TestShellBase' object has no attribute 'assertEqual' ============================================================================== 20 failed, 40 skipped in 0.87s =============================================================================== ```

[kislyuk]

Again, in the interest of clarity - this project does not support the use of pytest to run unittest style tests. Package manager maintainers who wish to run the test suite are expected to follow the package provided method of running tests. While pytest does a good job of emulating the basic unittest protocol, it does not implement the full fidelity of the unittest API and can't be expected to run tests reliably in suites that are not written to support pytest.

[kloczek]

Issue is that amongst +1.1k alredy packaged as rpm packages python modules probably +1/4 have been declared to support only unittest and despite that only on testing argcomplete unittest test suite it fails usimg pytest.

[kislyuk]

Perhaps you should stop trying to test it with pytest then?