Resolution of the absolute paths for posargs arguments fails when the tox root directory is located on a drive mapped with subst.
This happens, when e.g. a pytest test file is specified on the command line.
In my case, the sources are mapped to a Windows "drive" from inside a WSL distro (using subst W: \\wsl$\distro\path), but it also can be reproduced in Windows only.
The error is a combination of different aspects:
pathlib's resolve() is called for the root path in all cases, even when it is already absolute (config/main.py:101, make()).
resolve() does also resolve the subst drive letter to the original path, changing the drive and path prefix
not all paths are resolve()d during the processing of posargs
os.path.relpath() gets called to strip the prefix from posargs up to the tox root, but fails due to different drive letters and prefixes
Not resolving root when it already root.is_absolute() would work for my specific usecase, but could break if the root is mapped and relative
Resolving all paths for root and posargs could work, but:
Resolving the path at all may lead to other issues, if the real path is a UNC path, which was specifically mapped to a drive to avoid some issue with tools that don't handle UNC. There is a discussion about that at https://discuss.python.org/t/pathlib-absolute-vs-resolve/2573
Using mapped network drives (SMB share) doesn't have the same issue, it seems. resolve() does not expose the mapped location for that.
Issue
Resolution of the absolute paths for posargs arguments fails when the tox root directory is located on a drive mapped with subst.
This happens, when e.g. a pytest test file is specified on the command line. In my case, the sources are mapped to a Windows "drive" from inside a WSL distro (using
subst W: \\wsl$\distro\path
), but it also can be reproduced in Windows only.The error is a combination of different aspects:
make()
).Environment
Output of
```console Package Version ------------------------- ----------- annotated-types 0.5.0 attrs 23.1.0 black 23.7.0 build 0.10.0 CacheControl 0.12.14 cachetools 5.3.1 certifi 2023.7.22 chardet 5.2.0 charset-normalizer 3.2.0 cleo 2.0.1 click 8.1.5 colorama 0.4.6 crashtest 0.4.1 distlib 0.3.7 dulwich 0.21.5 filelock 3.12.2 flake8 6.0.0 html5lib 1.1 idna 3.4 importlib-metadata 6.8.0 installer 0.7.0 jaraco.classes 3.3.0 jsonschema 4.19.0 jsonschema-specifications 2023.7.1 keyring 23.13.1 lockfile 0.12.2 mccabe 0.7.0 more-itertools 10.1.0 msgpack 1.0.5 mypy 1.4.1 mypy-extensions 1.0.0 packaging 23.1 pathspec 0.11.1 pexpect 4.8.0 pip 23.2.1 pkginfo 1.9.6 platformdirs 3.10.0 pluggy 1.2.0 poetry 1.5.1 poetry-core 1.6.1 poetry-plugin-export 1.4.0 ptyprocess 0.7.0 pycodestyle 2.10.0 pydantic 2.0.3 pydantic_core 2.3.0 pyflakes 3.0.1 pyproject-api 1.5.3 pyproject_hooks 1.0.0 pywin32-ctypes 0.2.2 rapidfuzz 2.15.1 referencing 0.30.2 requests 2.31.0 requests-toolbelt 1.0.0 rpds-py 0.9.2 setuptools 65.5.0 shellingham 1.5.0.post1 six 1.16.0 tomlkit 0.12.1 tox 4.7.0 trove-classifiers 2023.7.6 typing_extensions 4.7.1 urllib3 1.26.16 virtualenv 20.24.2 webencodings 0.5.1 zipp 3.16.2 ```pip list
of the host Python, wheretox
is installedOutput of running tox
Output of
```console O:\tox_resolve_test\> tox -rvv -- test.py py: 345 W remove tox env folder C:\_work\py\tox_resolve_test\.tox\py [tox\tox_env\api.py:322] py: 601 I find interpreter for spec PythonSpec(path=C:\Toolchain\Python\Python311\python.exe) [virtualenv\discovery\builtin.py:58] py: 601 I proposed PythonInfo(spec=CPython3.11.4.final.0-64, exe=C:\Toolchain\Python\Python311\python.exe, platform=win32, version='3.11.4 (tags/v3.11.4:d2340ef, Jun 7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)]', encoding_fs_io=utf-8-utf-8) [virtualenv\discovery\builtin.py:65] py: 601 D accepted PythonInfo(spec=CPython3.11.4.final.0-64, exe=C:\Toolchain\Python\Python311\python.exe, platform=win32, version='3.11.4 (tags/v3.11.4:d2340ef, Jun 7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)]', encoding_fs_io=utf-8-utf-8) [virtualenv\discovery\builtin.py:67] py: 601 D symlink on filesystem does not work [virtualenv\info.py:45] py: 601 D filesystem is not case-sensitive [virtualenv\info.py:26] py: 665 I create virtual environment via CPython3Windows(dest=C:\_work\py\tox_resolve_test\.tox\py, clear=False, no_vcs_ignore=False, global=False) [virtualenv\run\session.py:50] py: 665 D create folder C:\_work\py\tox_resolve_test\.tox\py\Lib\site-packages [virtualenv\util\path\_sync.py:12] py: 665 D create folder C:\_work\py\tox_resolve_test\.tox\py\Scripts [virtualenv\util\path\_sync.py:12] py: 665 D write C:\_work\py\tox_resolve_test\.tox\py\pyvenv.cfg [virtualenv\create\pyenv_cfg.py:32] py: 665 D home = C:\Toolchain\Python\Python311 [virtualenv\create\pyenv_cfg.py:36] py: 665 D implementation = CPython [virtualenv\create\pyenv_cfg.py:36] py: 665 D version_info = 3.11.4.final.0 [virtualenv\create\pyenv_cfg.py:36] py: 665 D virtualenv = 20.24.2 [virtualenv\create\pyenv_cfg.py:36] py: 665 D include-system-site-packages = false [virtualenv\create\pyenv_cfg.py:36] py: 665 D base-prefix = C:\Toolchain\Python\Python311 [virtualenv\create\pyenv_cfg.py:36] py: 665 D base-exec-prefix = C:\Toolchain\Python\Python311 [virtualenv\create\pyenv_cfg.py:36] py: 665 D base-executable = C:\Toolchain\Python\Python311\python.exe [virtualenv\create\pyenv_cfg.py:36] py: 665 D copy C:\Toolchain\Python\Python311\Lib\venv\scripts\nt\python.exe to C:\_work\py\tox_resolve_test\.tox\py\Scripts\python.exe [virtualenv\util\path\_sync.py:40] py: 665 D copy C:\Toolchain\Python\Python311\Lib\venv\scripts\nt\pythonw.exe to C:\_work\py\tox_resolve_test\.tox\py\Scripts\pythonw.exe [virtualenv\util\path\_sync.py:40] py: 665 D create virtualenv import hook file C:\_work\py\tox_resolve_test\.tox\py\Lib\site-packages\_virtualenv.pth [virtualenv\create\via_global_ref\api.py:91] py: 665 D create C:\_work\py\tox_resolve_test\.tox\py\Lib\site-packages\_virtualenv.py [virtualenv\create\via_global_ref\api.py:94] py: 665 D ============================== target debug ============================== [virtualenv\run\session.py:52] py: 665 D debug via 'C:\_work\py\tox_resolve_test\.tox\py\Scripts\python.exe' 'C:\Toolchain\Python\Python311\Lib\site-packages\virtualenv\create\debug.py' [virtualenv\create\creator.py:200] py: 665 D { "sys": { "executable": "C:\\_work\\py\\tox_resolve_test\\.tox\\py\\Scripts\\python.exe", "_base_executable": "C:\\Toolchain\\Python\\Python311\\python.exe", "prefix": "C:\\_work\\py\\tox_resolve_test\\.tox\\py", "base_prefix": "C:\\Toolchain\\Python\\Python311", "real_prefix": null, "exec_prefix": "C:\\_work\\py\\tox_resolve_test\\.tox\\py", "base_exec_prefix": "C:\\Toolchain\\Python\\Python311", "path": [ "C:\\Toolchain\\Python\\Python311\\python311.zip", "C:\\Toolchain\\Python\\Python311\\DLLs", "C:\\Toolchain\\Python\\Python311\\Lib", "C:\\Toolchain\\Python\\Python311", "C:\\_work\\py\\tox_resolve_test\\.tox\\py", "C:\\_work\\py\\tox_resolve_test\\.tox\\py\\Lib\\site-packages" ], "meta_path": [ "tox -rvv -- test.py
on the subst driveOutput of
```console C:\_work\py\tox_resolve_test\>tox -rvv -- test.py py: 347 W remove tox env folder C:\_work\py\tox_resolve_test\.tox\py [tox\tox_env\api.py:322] py: 601 I find interpreter for spec PythonSpec(path=C:\Toolchain\Python\Python311\python.exe) [virtualenv\discovery\builtin.py:58] py: 601 I proposed PythonInfo(spec=CPython3.11.4.final.0-64, exe=C:\Toolchain\Python\Python311\python.exe, platform=win32, version='3.11.4 (tags/v3.11.4:d2340ef, Jun 7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)]', encoding_fs_io=utf-8-utf-8) [virtualenv\discovery\builtin.py:65] py: 601 D accepted PythonInfo(spec=CPython3.11.4.final.0-64, exe=C:\Toolchain\Python\Python311\python.exe, platform=win32, version='3.11.4 (tags/v3.11.4:d2340ef, Jun 7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)]', encoding_fs_io=utf-8-utf-8) [virtualenv\discovery\builtin.py:67] py: 601 D symlink on filesystem does not work [virtualenv\info.py:45] py: 616 D filesystem is not case-sensitive [virtualenv\info.py:26] py: 663 I create virtual environment via CPython3Windows(dest=C:\_work\py\tox_resolve_test\.tox\py, clear=False, no_vcs_ignore=False, global=False) [virtualenv\run\session.py:50] py: 663 D create folder C:\_work\py\tox_resolve_test\.tox\py\Lib\site-packages [virtualenv\util\path\_sync.py:12] py: 663 D create folder C:\_work\py\tox_resolve_test\.tox\py\Scripts [virtualenv\util\path\_sync.py:12] py: 663 D write C:\_work\py\tox_resolve_test\.tox\py\pyvenv.cfg [virtualenv\create\pyenv_cfg.py:32] py: 663 D home = C:\Toolchain\Python\Python311 [virtualenv\create\pyenv_cfg.py:36] py: 663 D implementation = CPython [virtualenv\create\pyenv_cfg.py:36] py: 663 D version_info = 3.11.4.final.0 [virtualenv\create\pyenv_cfg.py:36] py: 663 D virtualenv = 20.24.2 [virtualenv\create\pyenv_cfg.py:36] py: 663 D include-system-site-packages = false [virtualenv\create\pyenv_cfg.py:36] py: 663 D base-prefix = C:\Toolchain\Python\Python311 [virtualenv\create\pyenv_cfg.py:36] py: 663 D base-exec-prefix = C:\Toolchain\Python\Python311 [virtualenv\create\pyenv_cfg.py:36] py: 663 D base-executable = C:\Toolchain\Python\Python311\python.exe [virtualenv\create\pyenv_cfg.py:36] py: 663 D copy C:\Toolchain\Python\Python311\Lib\venv\scripts\nt\python.exe to C:\_work\py\tox_resolve_test\.tox\py\Scripts\python.exe [virtualenv\util\path\_sync.py:40] py: 663 D copy C:\Toolchain\Python\Python311\Lib\venv\scripts\nt\pythonw.exe to C:\_work\py\tox_resolve_test\.tox\py\Scripts\pythonw.exe [virtualenv\util\path\_sync.py:40] py: 679 D create virtualenv import hook file C:\_work\py\tox_resolve_test\.tox\py\Lib\site-packages\_virtualenv.pth [virtualenv\create\via_global_ref\api.py:91] py: 679 D create C:\_work\py\tox_resolve_test\.tox\py\Lib\site-packages\_virtualenv.py [virtualenv\create\via_global_ref\api.py:94] py: 679 D ============================== target debug ============================== [virtualenv\run\session.py:52] py: 679 D debug via 'C:\_work\py\tox_resolve_test\.tox\py\Scripts\python.exe' 'C:\Toolchain\Python\Python311\Lib\site-packages\virtualenv\create\debug.py' [virtualenv\create\creator.py:200] py: 679 D { "sys": { "executable": "C:\\_work\\py\\tox_resolve_test\\.tox\\py\\Scripts\\python.exe", "_base_executable": "C:\\Toolchain\\Python\\Python311\\python.exe", "prefix": "C:\\_work\\py\\tox_resolve_test\\.tox\\py", "base_prefix": "C:\\Toolchain\\Python\\Python311", "real_prefix": null, "exec_prefix": "C:\\_work\\py\\tox_resolve_test\\.tox\\py", "base_exec_prefix": "C:\\Toolchain\\Python\\Python311", "path": [ "C:\\Toolchain\\Python\\Python311\\python311.zip", "C:\\Toolchain\\Python\\Python311\\DLLs", "C:\\Toolchain\\Python\\Python311\\Lib", "C:\\Toolchain\\Python\\Python311", "C:\\_work\\py\\tox_resolve_test\\.tox\\py", "C:\\_work\\py\\tox_resolve_test\\.tox\\py\\Lib\\site-packages" ], "meta_path": [ "tox -rvv -- test.py
on the real pathMinimal example
tox.ini:
test.py:
subst mount: