7.0.0 regression: custom options handling

[zhihaoy]

• [x] a detailed description of the bug or problem you are having
• [x] output of pip list from the virtual environment you are using
• [x] pytest and operating system versions
• [ ] minimal example if possible

The option is added in using

def pytest_addoption(parser):
    parser.addoption("--engines_dir", action="store")

in conftest.py.

Result:

Traceback (most recent call last):
  File "c:\python\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\python\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "c:\python\lib\site-packages\pytest\__main__.py", line 5, in <module>
    raise SystemExit(pytest.console_main())
  File "c:\python\lib\site-packages\_pytest\config\__init__.py", line 188, in console_main
    code = main()
  File "c:\python\lib\site-packages\_pytest\config\__init__.py", line 146, in main
    config = _prepareconfig(args, plugins)
  File "c:\python\lib\site-packages\_pytest\config\__init__.py", line 325, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "c:\python\lib\site-packages\pluggy\_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "c:\python\lib\site-packages\pluggy\_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "c:\python\lib\site-packages\pluggy\_callers.py", line 55, in _multicall
    gen.send(outcome)
  File "c:\python\lib\site-packages\_pytest\helpconfig.py", line 102, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "c:\python\lib\site-packages\pluggy\_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "c:\python\lib\site-packages\pluggy\_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "c:\python\lib\site-packages\_pytest\config\__init__.py", line 1013, in pytest_cmdline_parse
    self.parse(args)
  File "c:\python\lib\site-packages\_pytest\config\__init__.py", line 1301, in parse
    self._preparse(args, addopts=addopts)
  File "c:\python\lib\site-packages\_pytest\config\__init__.py", line 1203, in _preparse
    self.hook.pytest_load_initial_conftests(
  File "c:\python\lib\site-packages\pluggy\_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "c:\python\lib\site-packages\pluggy\_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "c:\python\lib\site-packages\pluggy\_callers.py", line 60, in _multicall
    return outcome.get_result()
  File "c:\python\lib\site-packages\pluggy\_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "c:\python\lib\site-packages\pluggy\_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "c:\python\lib\site-packages\_pytest\config\__init__.py", line 1080, in pytest_load_initial_conftests
    self.pluginmanager._set_initial_conftests(
  File "c:\python\lib\site-packages\_pytest\config\__init__.py", line 525, in _set_initial_conftests
    if anchor.exists():  # we found some file object
  File "c:\python\lib\pathlib.py", line 1400, in exists
    self.stat()
  File "c:\python\lib\pathlib.py", line 1197, in stat
    return self._accessor.stat(self)
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: 'C:\\Some\\Where\\Pytest\\Starts\\--engines_dir=C:\\The\\Argument\\We\\Passed\\To\\This\\Option'

Installed packages:

Installing collected packages: ipaddress, iniconfig, distro, certifi, urllib3, tomli, pyparsing, py, pluggy, idna, colorama, charset-normalizer, attrs, atomicwrites, requests, packaging, pytest
Successfully installed atomicwrites-1.4.0 attrs-21.4.0 certifi-2021.10.8 charset-normalizer-2.0.11 colorama-0.4.4 distro-1.6.0 idna-3.3 iniconfig-1.1.1 ipaddress-1.0.23 packaging-21.3 pluggy-1.0.0 py-1.11.0 pyparsing-3.0.7 pytest-7.0.0 requests-2.27.1 tomli-2.0.0 urllib3-1.26.8

OS: Windows 10 Enterprise TH2 SP1 64bit

[bluetech]

Can you show the exact way you invoke pytest?

[zhihaoy]

Can you show the exact way you invoke pytest?

Passing this array to subprocess.call

['c:\\python\\python.exe', '-m', 'pytest', '-v', '--some_custom_option', '--many_other_custom_options', '--engines_dir=C:\\The\\Argument\\We\\Passed\\To\\This\\Option', '-k', 'key1 or key2', '-m', 'not criteria1']

I haven't tried running this command in cmd, but I doubt that will make a difference.

[bluetech]

If you split '--engines_dir=C:\\The\\Argument\\We\\Passed\\To\\This\\Option' into two separate arguments '--engines_dir', 'C:\\The\\Argument\\We\\Passed\\To\\This\\Option' does it still happen?

[RonnyPfannschmidt]

Please investigate what's happening if you pass the path value of the arguments to pathlib.Path

As the install location of the python doesn't match the filename I'd expect, details on what type of python this is would be highly appreciated

[zhihaoy]

If you split '--engines_dir=C:\\The\\Argument\\We\\Passed\\To\\This\\Option' into two separate arguments '--engines_dir', 'C:\\The\\Argument\\We\\Passed\\To\\This\\Option' does it still happen?

No, this works without an issue. However, I must convert all the options to use this style, otherwise, I get an error from pytest's __main__.

Please investigate what's happening if you pass the path value of the arguments to pathlib.Path

Nothing special, the path is a valid path, and I got a valid WindowsPath back. The defective paths have spaces inside, though.

As the install location of the python doesn't match the filename I'd expect, details on what type of python this is would be highly appreciated

An official Python 3.8.8 installer was used, installed to a non-default location.

Now I figured out when the issue happens. It only reproduces with Python 3.8.8. It does not reproduce with Python 3.8.10. The install location / whether virtualenv is used does not matter.

[phdru]

I have a similar or exactly this problem. I prepared a small test case: https://github.com/phdru/pytest3-w64-conftest . It works on Linux, it works on Windows with Python 2.7 and 3.5, but started to fail on Windows with Python 3.6+. This is a test run: https://github.com/phdru/pytest3-w64-conftest/actions/runs/3074106082 . The command (ran from tox) is pytest -D "mysql://localhost/test?fest=pest" . The full traceback is:

Traceback (most recent call last):
  File "c:\hostedtoolcache\windows\python\3.7.9\x64\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\hostedtoolcache\windows\python\3.7.9\x64\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\Scripts\pytest.EXE\__main__.py", line 7, in <module>
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\lib\site-packages\_pytest\config\__init__.py", line 187, in console_main
    code = main()
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\lib\site-packages\_pytest\config\__init__.py", line 145, in main
    config = _prepareconfig(args, plugins)
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\lib\site-packages\_pytest\config\__init__.py", line 325, in _prepareconfig
    pluginmanager=pluginmanager, args=args
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\lib\site-packages\pluggy\_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\lib\site-packages\pluggy\_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\lib\site-packages\pluggy\_callers.py", line 55, in _multicall
    gen.send(outcome)
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\lib\site-packages\_pytest\helpconfig.py", line 102, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\lib\site-packages\pluggy\_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\lib\site-packages\pluggy\_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\lib\site-packages\_pytest\config\__init__.py", line 1017, in pytest_cmdline_parse
    self.parse(args)
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\lib\site-packages\_pytest\config\__init__.py", line 1305, in parse
    self._preparse(args, addopts=addopts)
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\lib\site-packages\_pytest\config\__init__.py", line 1208, in _preparse
    early_config=self, args=args, parser=self._parser
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\lib\site-packages\pluggy\_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\lib\site-packages\pluggy\_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\lib\site-packages\pluggy\_callers.py", line 60, in _multicall
    return outcome.get_result()
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\lib\site-packages\pluggy\_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\lib\site-packages\pluggy\_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\lib\site-packages\_pytest\config\__init__.py", line 1085, in pytest_load_initial_conftests
    early_config.known_args_namespace, rootpath=early_config.rootpath
  File "D:\a\pytest3-w64-conftest\pytest3-w64-conftest\.tox\py37\lib\site-packages\_pytest\config\__init__.py", line 528, in _set_initial_conftests
    if anchor.exists():  # we found some file object
  File "c:\hostedtoolcache\windows\python\3.7.9\x64\lib\pathlib.py", line 1361, in exists
    self.stat()
  File "c:\hostedtoolcache\windows\python\3.7.9\x64\lib\pathlib.py", line 1183, in stat
    return self._accessor.stat(self)
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: 'D:\\a\\pytest3-w64-conftest\\pytest3-w64-conftest\\mysql:\\localhost\\test?fest=pest'

[phdru]

The test passed with pytest < 7.0. Code, test run.

Failed with pytest == 7.0. Code, run.

So the bug was introduced between 6.2.5 and 7.0.