search

nedbat / coveragepy

The code coverage tool for Python
https://coverage.readthedocs.io
Apache License 2.0
3.01k stars 432 forks source link

Unquoted path in command generated in `venv_world_fixture` causes error if path contains spaces #1374

Closed SebastiaanZ closed 2 years ago

SebastiaanZ commented 2 years ago

Describe the bug The venv_world_fixture creates a string that will be run as a command-line command. The last argument in this generated command is a path to the project or a wheel (see commit d7a7f897 for context):

https://github.com/nedbat/coveragepy/blob/b3aa805c064f6c07d0780c815d92960484999afc/tests/test_venv.py#L141-L147

As the path COVERAGE_INSTALL_ARGS is not quoted, this command will fail if the path contains spaces.

To Reproduce How can we reproduce the problem? Please be specific. Don't link to a failing CI job. Answer the questions below:

  1. What version of Python are you using? 3.7
  2. What version of coverage.py shows the problem? The latest version on main for development purposes. Output of coverage debug sys:
(coveragepy) C:\Users\sze\OneDrive - Ordina\Documenten\coveragepy>python -m coverage debug sys
-- sys -------------------------------------------------------
               coverage_version: 6.3.4a0                                                                       
                coverage_module: C:\Users\sze\OneDrive - Ordina\Documenten\coveragepy\coverage\__init__.py
                         tracer: -none-                                                                        
                        CTracer: available                                                                     
           plugins.file_tracers: -none-                                                                        
            plugins.configurers: -none-                                                                        
      plugins.context_switchers: -none-                                                                        
              configs_attempted: .coveragerc                                                                   
                                 setup.cfg                                                                     
                                 tox.ini                                                                       
                                 pyproject.toml
                   configs_read: C:\Users\sze\OneDrive - Ordina\Documenten\coveragepy\setup.cfg
                                 C:\Users\sze\OneDrive - Ordina\Documenten\coveragepy\tox.ini
                    config_file: None
                config_contents: -none-
                      data_file: -none-
                         python: 3.7.9 (tags/v3.7.9:13c94747c7, Aug 17 2020, 18:58:18) [MSC v.1900 64 bit (AMD64)]
                       platform: Windows-10-10.0.19041-SP0
                 implementation: CPython
                     executable: C:\Users\sze\.virtualenvs\coveragepy\Scripts\python.exe
                   def_encoding: utf-8
                    fs_encoding: utf-8
                            pid: 8856
                            cwd: C:\Users\sze\OneDrive - Ordina\Documenten\coveragepy
                           path: C:\Users\sze\OneDrive - Ordina\Documenten\coveragepy
                                 C:\Users\sze\AppData\Local\Programs\Python\Python37\python37.zip
                                 C:\Users\sze\AppData\Local\Programs\Python\Python37\DLLs
                                 C:\Users\sze\AppData\Local\Programs\Python\Python37\lib
                                 C:\Users\sze\AppData\Local\Programs\Python\Python37
                                 C:\Users\sze\.virtualenvs\coveragepy
                                 C:\Users\sze\.virtualenvs\coveragepy\lib\site-packages
                    environment: TEMP = C:\Users\sze\AppData\Local\Temp
                                 TMP = C:\Users\sze\AppData\Local\Temp
                   command_line: C:\Users\sze\OneDrive - Ordina\Documenten\coveragepy\coverage\__main__.py debug sys
                sqlite3_version: 2.6.0
         sqlite3_sqlite_version: 3.31.1
             sqlite3_temp_store: 0
        sqlite3_compile_options: COMPILER=msvc-1900; ENABLE_FTS4; ENABLE_FTS5
                                 THREADSAFE=1

  1. What versions of what packages do you have installed? The output of pip freeze is helpful.

    astroid==2.11.5
atomicwrites==1.4.0
attrs==21.4.0
backports.functools-lru-cache==1.6.4
bleach==5.0.0
build==0.7.0
certifi==2021.10.8
charset-normalizer==2.0.12
check-manifest==0.48
cogapp==3.3.0
colorama==0.4.4
commonmark==0.9.1
decorator==5.1.1
dill==0.3.4
distlib==0.3.4
docutils==0.17.1
execnet==1.9.0
filelock==3.6.0
flaky==3.7.0
future==0.18.2
greenlet==1.1.2
hypothesis==6.46.3
idna==3.3
importlib-metadata==4.11.3
iniconfig==1.1.1
isort==5.10.1
jedi==0.18.1
keyring==23.5.0
lazy-object-proxy==1.7.1
libsass==0.21.0
mccabe==0.7.0
packaging==21.3
parso==0.8.3
pep517==0.12.0
pkginfo==1.8.2
platformdirs==2.5.2
pluggy==1.0.0
pudb==2022.1.1
py==1.11.0
PyContracts @ git+https://github.com/slorg1/contracts@c5a6da27d4dc9985f68e574d20d86000880919c3
Pygments==2.12.0
pylint==2.13.8
tox==3.25.0
twine==4.0.0
typed-ast==1.5.3
typing_extensions==4.2.0
urllib3==1.26.9
urwid==2.1.2
urwid-readline==0.13
virtualenv==20.14.1
webencodings==0.5.1
wrapt==1.14.1
zipp==3.8.0

  2. What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix.

L146 in the following snippet: https://github.com/nedbat/coveragepy/blob/b3aa805c064f6c07d0780c815d92960484999afc/tests/test_venv.py#L141-L147

We tried a hotfix/hack by quoting the path here to further investigate the problem (f'"{COVERAGE_INSTALL_ARGS }"'), which worked but led to a failing test (probably because of double escaping):

[gw0] [100%] FAILED test_venv.py::VirtualenvTest::test_third_party_venv_isnt_measured[python -m coverage] 
tests\test_venv.py:202 (VirtualenvTest.test_third_party_venv_isnt_measured[python -m coverage])
self = <tests.test_venv.VirtualenvTest object at 0x000001BB016C16C8>
coverage_command = 'python -m coverage'

    def test_third_party_venv_isnt_measured(self, coverage_command):
        out = run_in_venv(coverage_command + " run --source=. myproduct.py")
        # In particular, this warning doesn't appear:
        # Already imported a file that will be measured: .../coverage/__main__.py
        assert out == self.expected_stdout

        # Check that our tracing was accurate. Files are mentioned because
        # --source refers to a file.
        debug_out = self.get_trace_output()
>       assert re_lines(
            r"^Not tracing .*\bexecfile.py': inside --source, but is third-party",
            debug_out,
        )
E       assert []
E        +  where [] = re_lines("^Not tracing .*\\bexecfile.py': inside --source, but is third-party", "sys.path:\n    C:\\Users\\sze\\AppData\\Local\\Temp\\pytest-of-sze\\pytest-14\\popen-gw0\\venv_world0\n    ...sze\\\\OneDrive - Ordina\\\\Documenten\\\\coveragepy\\\\coverage\\\\collector.py': falls outside the --source spec\n")

C:\Users\sze\OneDrive - Ordina\Documenten\coveragepy\tests\test_venv.py:212: AssertionError
  1. What commands did you run?

tox

Expected behavior The test suite runs without errors (and hopefully failures) for a freshly cloned repository, even if the path to the cloned repo contains spaces.

Additional context We were doing an open-source mobbing session at work (Ordina Pythoneers) and decided to try and fix another bug. We encountered this while setting up the repository.

nedbat commented 2 years ago

When I try cloning into "cov py", I get 82 failed tests and 10 errored tests! Is that what you saw? This is with Python 3.7 on a Mac:

FAILED tests/test_concurrency.py::ConcurrencyTest::test_eventlet - assert '/bin/sh: /pr...r directory\n' == '499500\n'
FAILED tests/test_concurrency.py::ConcurrencyTest::test_threads - assert '/bin/sh: /pr...r directory\n' == '499500\n'
FAILED tests/test_concurrency.py::ConcurrencyTest::test_eventlet_simple_code - assert '/bin/sh: /pr...r directory\n' == '499500\n'
FAILED tests/test_concurrency.py::ConcurrencyTest::test_greenlet_simple_code - assert '/bin/sh: /pr...r directory\n' == '499500\n'
FAILED tests/test_concurrency.py::ConcurrencyTest::test_gevent - assert '/bin/sh: /pr...r directory\n' == '499500\n'
FAILED tests/test_concurrency.py::ConcurrencyTest::test_threads_simple_code - assert '/bin/sh: /pr...r directory\n' == '499500\n'
FAILED tests/test_concurrency.py::ConcurrencyTest::test_gevent_simple_code - assert '/bin/sh: /pr...r directory\n' == '499500\n'
FAILED tests/test_concurrency.py::ConcurrencyTest::test_greenlet - assert '/bin/sh: /pr...r directory\n' == 'hello world\n42\n'
FAILED tests/test_concurrency.py::ConcurrencyTest::test_threads_with_gevent - assert '/bin/sh: /pr...r directory\n' == 'done\n'
FAILED tests/test_concurrency.py::ConcurrencyTest::test_bug_330 - assert '/bin/sh: /pr...r directory\n' == '0\n'
FAILED tests/test_concurrency.py::MultiprocessingTest::test_multiprocessing_bootstrap_error_handling - assert 'Exception during multiprocessing bootstrap init' in '/bi...
FAILED tests/test_concurrency.py::MultiprocessingTest::test_multiprocessing_with_branching[fork] - assert '/bin/sh: /pr... or directory' == '3 pids, total = 92695'
FAILED tests/test_concurrency.py::MultiprocessingTest::test_bug_890 - assert '/bin/sh: /pr... or directory' == 'ok'
FAILED tests/test_concurrency.py::MultiprocessingTest::test_multiprocessing_simple[spawn] - assert '/bin/sh: /pr... or directory' == '3 pids, total = 92695'
FAILED tests/test_concurrency.py::MultiprocessingTest::test_multiprocessing_simple[fork] - assert '/bin/sh: /pr... or directory' == '3 pids, total = 92695'
FAILED tests/test_concurrency.py::MultiprocessingTest::test_multiprocessing_append[fork] - assert '/bin/sh: /pr... or directory' == '3 pids, total = 92695'
FAILED tests/test_concurrency.py::MultiprocessingTest::test_multiprocessing_append[spawn] - assert '/bin/sh: /pr... or directory' == '3 pids, total = 92695'
FAILED tests/test_concurrency.py::MultiprocessingTest::test_multiprocessing_and_gevent[fork] - assert '/bin/sh: /pr... or directory' == '3 pids, total = 47251750'
FAILED tests/test_concurrency.py::MultiprocessingTest::test_multiprocessing_and_gevent[spawn] - assert '/bin/sh: /pr... or directory' == '3 pids, total = 47251750'
FAILED tests/test_concurrency.py::SigtermTest::test_sigterm_still_runs - assert '/bin/sh: /pr...r directory\n' == 'START\nSIGTERM\nEND\n'
FAILED tests/test_concurrency.py::SigtermTest::test_sigterm_saves_data - assert '/bin/sh: /pr...r directory\n' == 'START\nNOT THREE\nEND\n'
FAILED tests/test_concurrency.py::MultiprocessingTest::test_multiprocessing_with_branching[spawn] - assert '/bin/sh: /pr... or directory' == '3 pids, total = 92695'
FAILED tests/test_phystokens.py::Bug529Test::test_bug_529 - assert 126 == 0
FAILED tests/test_plugins.py::PluginTest::test_local_files_are_importable - assert '/bin/sh: /pr...r directory\n' == 'MAIN\n'
FAILED tests/test_oddball.py::MockingProtectionTest::test_os_path_exists - assert '/bin/sh: /pr...r directory\n' == 'in test\nbug....py\n23\n17\n'
FAILED tests/test_process.py::ProcessTest::test_tests_dir_is_importable - AssertionError: File '.coverage' should exist
FAILED tests/test_process.py::ProcessTest::test_save_on_exit - AssertionError: File '.coverage' should exist
FAILED tests/test_process.py::ProcessTest::test_running_missing_file - assert None
FAILED tests/test_process.py::ProcessTest::test_append_data_with_different_file - assert '/bin/sh: /pr...r directory\n' == 'done\n'
FAILED tests/test_process.py::ProcessTest::test_combine_with_aliases - assert '/bin/sh: /pr...r directory\n' == '1 2\n'
FAILED tests/test_process.py::ProcessTest::test_combine_with_rc - assert '/bin/sh: /pr...r directory\n' == 'done\n'
FAILED tests/test_process.py::ProcessTest::test_erase_parallel - AssertionError: File 'data.dat' shouldn't exist
FAILED tests/test_process.py::ProcessTest::test_missing_source_file - assert None
FAILED tests/test_process.py::ProcessTest::test_append_data - assert '/bin/sh: /pr...r directory\n' == 'done\n'
FAILED tests/test_process.py::ProcessTest::test_fork - assert '/bin/sh: /pr...r directory\n' == 'Child!\n'
FAILED tests/test_process.py::ProcessTest::test_warns_if_never_run - assert "No file to run: 'i_dont_exist.py'" in '/bin/sh: /private/tmp/my dir/coveragepy/.tox/py37/b...
FAILED tests/test_process.py::ProcessTest::test_warnings_trace_function_changed_with_threads - assert 'Hello\n' in '/bin/sh: /private/tmp/my dir/coveragepy/.tox/py37/b...
FAILED tests/test_process.py::ProcessTest::test_lang_c - assert '/bin/sh: /pr...r directory\n' == '1\n2\n'
FAILED tests/test_process.py::ProcessTest::test_warning_trace_function_changed - assert 'Hello\n' in '/bin/sh: /private/tmp/my dir/coveragepy/.tox/py37/bin/coverage: "...
FAILED tests/test_process.py::ProcessTest::test_coverage_run_envvar_is_in_coveragerun - assert '/bin/sh: /pr...r directory\n' == 'true\n'
FAILED tests/test_process.py::ProcessTest::test_append_can_create_a_data_file - assert '/bin/sh: /pr...r directory\n' == 'done\n'
FAILED tests/test_process.py::ProcessTest::test_code_exits_no_arg - assert '/bin/sh: /pr...r directory\n' == 'about to exit quietly..\n'
FAILED tests/test_process.py::ProcessTest::test_timid - assert '/bin/sh: /pr...r directory\n' == 'CTracer\n'
FAILED tests/test_process.py::ProcessTest::test_code_throws - assert '/bin/sh: /pr...r directory\n' == 'Traceback (m...ption: hey!\n'
FAILED tests/test_process.py::EnvironmentTest::test_coverage_run_is_like_python - assert '"DATA": "xyzzy"' in '/bin/sh: /private/tmp/my dir/coveragepy/.tox/py37/bin/co...
FAILED tests/test_process.py::ProcessTest::test_run_twice - AssertionError: assert 'Run 1\nRun 2...t-measured)\n' == 'Run 1\nRun 2...t-measured)\n'
FAILED tests/test_process.py::AliasedCommandTest::test_specific_alias_works - assert 'Code coverage for Python' in '/bin/sh: /private/tmp/my dir/coveragepy/.tox/py37/b...
FAILED tests/test_process.py::AliasedCommandTest::test_aliases_used_in_messages[coverage] - assert "Unknown command: 'foobar'" in '/bin/sh: /private/tmp/my dir/coverag...
FAILED tests/test_process.py::ProcessTest::test_code_exits - assert '/bin/sh: /pr...r directory\n' == 'about to exit..\n'
FAILED tests/test_process.py::EnvironmentTest::test_coverage_run_dashm_is_like_python_dashm - assert '"DATA": "xyzzy"' in '/bin/sh: /private/tmp/my dir/coveragepy/.tox...
FAILED tests/test_process.py::AliasedCommandTest::test_aliases_used_in_messages[coverage3] - assert "Unknown command: 'foobar'" in '/bin/sh: /private/tmp/my dir/covera...
FAILED tests/test_process.py::EnvironmentTest::test_coverage_run_far_away_is_like_python - assert '"DATA": "xyzzy"' in '/bin/sh: /private/tmp/my dir/coveragepy/.tox/py...
FAILED tests/test_process.py::AliasedCommandTest::test_aliases_used_in_messages[coverage-3.7] - assert "Unknown command: 'foobar'" in '/bin/sh: /private/tmp/my dir/cov...
FAILED tests/test_process.py::EnvironmentTest::test_bug_862 - AssertionError: assert 'inside foo\n' == '/bin/sh: som...r directory\n'
FAILED tests/test_process.py::EnvironmentTest::test_coverage_run_dashm_is_like_python_dashm_off_path - assert '"DATA": "xyzzy"' in '/bin/sh: /private/tmp/my dir/covera...
FAILED tests/test_process.py::EnvironmentTest::test_coverage_run_dashm_superset_of_doubledashsource - assert '"DATA": "xyzzy"' in '/bin/sh: /private/tmp/my dir/coverag...
FAILED tests/test_process.py::EnvironmentTest::test_coverage_run_dashm_equal_to_doubledashsource - assert '"DATA": "xyzzy"' in '/bin/sh: /private/tmp/my dir/coveragepy...
FAILED tests/test_process.py::EnvironmentTest::test_bug_909 - assert 'Init\nThe code\n' == '/bin/sh: /pr...r directory\n'
FAILED tests/test_process.py::FailUnderTest::test_report_43_is_ok - assert 126 == 0
FAILED tests/test_process.py::FailUnderNoFilesTest::test_report - assert 'No data to report.' in '/bin/sh: /private/tmp/my dir/coveragepy/.tox/py37/bin/coverage: "/pri...
FAILED tests/test_process.py::EnvironmentTest::test_coverage_run_dashm_dir_no_init_is_like_python - assert '"DATA": "xyzzy"' in '/bin/sh: /private/tmp/my dir/coveragep...
FAILED tests/test_process.py::YankedDirectoryTest::test_removing_directory_with_error - assert False
FAILED tests/test_process.py::AliasedCommandTest::test_major_version_works - assert 'Code coverage for Python' in '/bin/sh: /private/tmp/my dir/coveragepy/.tox/py37/bi...
FAILED tests/test_process.py::EnvironmentTest::test_coverage_run_dashm_dir_with_init_is_like_python - assert '"DATA": "xyzzy"' in '/bin/sh: /private/tmp/my dir/coverag...
FAILED tests/test_process.py::ProcessStartupTest::test_subprocess_with_pth_files_and_parallel - FileNotFoundError: [Errno 2] No such file or directory: 'out.txt'
FAILED tests/test_process.py::FailUnderTest::test_report_42p86_is_not_ok - assert 126 == 2
FAILED tests/test_process.py::EnvironmentTest::test_coverage_run_script_imports_doubledashsource - assert '"DATA": "xyzzy"' in '/bin/sh: /private/tmp/my dir/coveragepy...
FAILED tests/test_process.py::EnvironmentTest::test_coverage_zip_is_like_python - assert '"DATA": "xyzzy"' in "python: can't open file '/private/tmp/my': [Errno 2] No ...
FAILED tests/test_process.py::FailUnderTest::test_report_99p9_is_not_ok - assert 126 == 2
FAILED tests/test_process.py::ExcepthookTest::test_excepthook - assert 126 == 1
FAILED tests/test_process.py::FailUnderEmptyFilesTest::test_report - assert 126 == 0
FAILED tests/test_process.py::ExcepthookTest::test_excepthook_exit - assert 126 == 0
FAILED tests/test_process.py::YankedDirectoryTest::test_removing_directory - assert '/bin/sh: /pr...r directory\n' == 'noerror\n'
FAILED tests/test_summary.py::SummaryTest::test_report_wildcard - assert 1 == 5
FAILED tests/test_summary.py::SummaryTest::test_run_omit_vs_report_omit - AssertionError: assert 'covmod1.py' in []
FAILED tests/test_process.py::EnvironmentTest::test_coverage_run_dashm_is_like_python_dashm_with__main__207 - assert 'init\nmain\n' == '/bin/sh: /pr...r directory\n'
FAILED tests/test_process.py::ExcepthookTest::test_excepthook_throw - assert 126 == 1
FAILED tests/test_summary.py::SummaryTest::test_report_with_chdir - assert '/bin/sh: /pr...r directory\n' == 'Line One\nLine Two\nhello\n'
FAILED tests/test_process.py::FailUnderTest::test_report_43_is_not_ok - assert 126 == 2
FAILED tests/test_summary.py::SummaryTest::test_report_skip_covered_no_branches - assert '/bin/sh: /pr...r directory\n' == 'z\n'
FAILED tests/test_testing.py::CoverageTestTest::test_sub_python_is_this_python - assert 0 == 1
FAILED tests/test_process.py::EnvironmentTest::test_coverage_run_dir_is_like_python_dir - assert '"DATA": "xyzzy"' in '/bin/sh: /private/tmp/my dir/coveragepy/.tox/py3...
ERROR tests/test_venv.py::VirtualenvTest::test_third_party_venv_isnt_measured[coverage] - assert 1 == 0
ERROR tests/test_venv.py::VirtualenvTest::test_third_party_venv_isnt_measured[python -m coverage] - assert 1 == 0
ERROR tests/test_venv.py::VirtualenvTest::test_us_in_venv_isnt_measured[coverage] - assert 1 == 0
ERROR tests/test_venv.py::VirtualenvTest::test_us_in_venv_isnt_measured[python -m coverage] - assert 1 == 0
ERROR tests/test_venv.py::VirtualenvTest::test_venv_isnt_measured[coverage] - assert 1 == 0
ERROR tests/test_venv.py::VirtualenvTest::test_venv_isnt_measured[python -m coverage] - assert 1 == 0
ERROR tests/test_venv.py::VirtualenvTest::test_venv_with_dynamic_plugin[coverage] - assert 1 == 0
ERROR tests/test_venv.py::VirtualenvTest::test_venv_with_dynamic_plugin[python -m coverage] - assert 1 == 0
ERROR tests/test_venv.py::VirtualenvTest::test_installed_namespace_packages[coverage] - assert 1 == 0
ERROR tests/test_venv.py::VirtualenvTest::test_installed_namespace_packages[python -m coverage] - assert 1 == 0
ERROR tests/test_venv.py::VirtualenvTest::test_bug_888[coverage] - assert 1 == 0
ERROR tests/test_venv.py::VirtualenvTest::test_bug_888[python -m coverage] - assert 1 == 0
82 failed, 1050 passed, 10 skipped, 2 xfailed, 12 errors in 29.65s
nedbat commented 2 years ago

I can't even run coverage from the tox directory:

$ .tox/py37/bin/coverage
bash: .tox/py37/bin/coverage: "/private/tmp/my: bad interpreter: No such file or directory
$ head -3 .tox/py37/bin/coverage
#!"/private/tmp/my dir/coveragepy/.tox/py37/bin/python"
# EASY-INSTALL-ENTRY-SCRIPT: 'coverage','console_scripts','coverage'
import re
SebastiaanZ commented 2 years ago

We saw fewer error tests and a few failures. Running the tests with tox worked, but the fixture/test setup failed with a traceback pointing to the venv fixture. I'll run tox again and copy-paste the full output (later today).

nedbat commented 2 years ago

Feel free to re-open this if you get more information.