pytest fails to parse command line options with spaces

[Anton3]

So it's basically impossible to pass arguments containing spaces to pytest.

Bug reproducer

I actually invoke pytest using pytest.main, but the bug should probably reproduce via the standard command line invocation as well.

# runtests.py
pytest.main([
    '--foo-path=/home/some/path with/spaces',
    '--bar-path=/home/another path/with/spaces',
    '--baz=nospaces',
    'test/directory/path',
])

The options are registered as follows:

def pytest_addoption(parser) -> None:
    group = parser.getgroup('my-config')
    group.addoption('--foo-path', type=pathlib.Path)
    group.addoption('--bar-path', type=pathlib.Path)
    group.addoption('--baz')

Error message:

ERROR: usage: runtests [options] [file_or_dir] [file_or_dir] [...]
runtests: error: unrecognized arguments: --bar-path=/home/another path/with/spaces --baz=nospaces

pytest gets confused about all the args after the arg with spaces. It likely misinterprets with/spaces as the start of the list of tests, so from that POV all the following options are garbage.

I've also tried various escaping that works with shells, nothing seems to work.

My environment

Ubuntu 22.04.1 Python 3.10.12

pytest                8.3.2
pytest-aiohttp        1.0.5
pytest-asyncio        0.21.2

[The-Compiler]

Your reproducer runs fine here, and I don't see why it shouldn't, other than the missing imports. Given those, did you run your reproducer to see if it actually reproduces the problem? If so, could you show a shell session showing the (complete) files and what exactly you're running how?

[Anton3]

Here is a more complete example:

import os
import sys

import pytest

TESTSUITE_PYTHONPATH = ['/home/antonyzhilin/userver/testsuite/pytest_plugins']
TESTSUITE_PYTEST_ARGS = ['--build-dir=/home/antonyzhilin/ide/userver-cmake/debug '
 'spaces/core/functional_tests/basic_chaos',
 '--service-logs-file=/home/antonyzhilin/ide/userver-cmake/debug '
 'spaces/core/functional_tests/basic_chaos/Testing/Temporary/service.log',
 '--service-config=/home/antonyzhilin/userver/core/functional_tests/basic_chaos/static_config.yaml',
 '--service-source-dir=/home/antonyzhilin/userver/core/functional_tests/basic_chaos',
 '--service-binary=/home/antonyzhilin/ide/userver-cmake/debug '
 'spaces/core/functional_tests/basic_chaos/userver-core-tests-basic-chaos']

def testsuite_runner():
    args = [
        *TESTSUITE_PYTEST_ARGS,
        *sys.argv[1:],
    ]
    sys.path.extend(TESTSUITE_PYTHONPATH)
    os.environ['PATH'] = os.path.dirname('/home/antonyzhilin/ide/userver-cmake/debug spaces/venv-userver-testenv/bin/python') + ':' + os.environ['PATH']
    return pytest.main(args=args)

if __name__ == '__main__':
    sys.exit(testsuite_runner())

(This tests runner script is code-generated.)

The script is invoked as follows:

cd "/home/antonyzhilin/ide/userver-cmake/debug spaces/core/functional_tests/basic_chaos" && "/home/antonyzhilin/ide/userver-cmake/debug spaces/venv-userver-testenv/bin/python" "/home/antonyzhilin/ide/userver-cmake/debug spaces/core/functional_tests/basic_chaos/runtests-userver-core-tests-basic-chaos-tests-deadline" --service-logs-pretty --service-runner-mode -vvs /home/antonyzhilin/userver/core/functional_tests/basic_chaos/tests-deadline

[Anton3]

I've been able to work around the issue by putting the tests directory as the first argument using args[::-1]

[RonnyPfannschmidt]

This Looks like a confusion around test paths happens depending on whether a path, or a not yet known argument was passed

We should verify if it only happens if the single item variant is affected or whether it also happens if argument name and value are passed as multiple lists items

I am under the impression that the names of the initial conftests aren't determined in a stable manner in your case

[The-Compiler]

It's still unclear to me if this is an issue in pytest, or your tooling around it. Once again, a self-contained, runnable example that reproduces the issue but is complete enough so others can run it too would be very helpful.

[github-actions[bot]]

This issue is stale because it has the status: needs information label and requested follow-up information was not provided for 14 days.

[github-actions[bot]]

This issue was closed because it has the status: needs information label and follow-up information has not been provided for 7 days since being marked as stale.