Open jaimergp opened 2 years ago
There's also a more general issue with Windows path lengths. Please see issue ( https://github.com/conda/conda/issues/7203 ).
Hi there, thank you for your contribution!
This issue has been automatically marked as stale because it has not had recent activity. It will be closed automatically if no further activity occurs.
If you would like this issue to remain open please:
NOTE: If this issue was closed prematurely, please leave a comment.
Thanks!
This is still an issue today, and can be seen on this PR, which fails to ripgrep
(at least) this chestnut:
C:/bld/verapdf_1697427442234/_h_env/vpdf/profiles/veraPDF-validation-profiles-integration/PDF_A/2u/6.2 Graphics/6.2.11 Fonts/6.2.11.7 Unicode character maps/6.2.11.7.2 Level A and Level U conformance/verapdf-profile-6-2-11-7-2-t02.xml
With this error, now exposed more fully with conda-forge
's more-verbose logging setup:
Resource usage statistics from building verapdf:
Process count: 6
CPU time: Sys=0:00:21.8, User=0:01:30.0
Memory: 539.1M
Disk usage: 70.4M
Time elapsed: 0:03:39.0
INFO:conda_build.build:Packaging verapdf
Packaging verapdf
INFO:conda_build.build:Packaging verapdf-1.25.73-h57928b3_0
Packaging verapdf-1.25.73-h57928b3_0
number of files: 538
Fixing permissions
Packaged license file/s.
Traceback (most recent call last):
File "C:\Miniforge\Scripts\conda-mambabuild-script.py", line 9, in <module>
sys.exit(main())
File "C:\Miniforge\lib\site-packages\boa\cli\mambabuild.py", line 256, in main
call_conda_build(action, config)
File "C:\Miniforge\lib\site-packages\boa\cli\mambabuild.py", line 228, in call_conda_build
result = api.build(
File "C:\Miniforge\lib\site-packages\conda_build\api.py", line 253, in build
return build_tree(
File "C:\Miniforge\lib\site-packages\conda_build\build.py", line 3799, in build_tree
packages_from_this = build(
File "C:\Miniforge\lib\site-packages\conda_build\build.py", line 2875, in build
newly_built_packages = bundlers[pkg_type](output_d, m, env, stats)
File "C:\Miniforge\lib\site-packages\conda_build\build.py", line 2017, in bundle_conda
output["checksums"] = create_info_files(
File "C:\Miniforge\lib\site-packages\conda_build\build.py", line 1553, in create_info_files
files_with_prefix = get_files_with_prefix(m, replacements, files, prefix)
File "C:\Miniforge\lib\site-packages\conda_build\build.py", line 1171, in get_files_with_prefix
pfx_matches = have_regex_files(
File "C:\Miniforge\lib\site-packages\conda_build\build.py", line 653, in have_regex_files
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Miniforge\lib\subprocess.py", line 1456, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 206] The filename or extension is too long
Presumably, when found to be on windows, regex_files_rg
could prepend r"\\?\"
to any paths, as is done in a number of other libraries' test tools, such as pytest.
def ensure_extended_length_path(path: Path) -> Path:
"""Get the extended-length version of a path (Windows).
On Windows, by default, the maximum length of a path (MAX_PATH) is 260
characters, and operations on paths longer than that fail. But it is possible
to overcome this by converting the path to "extended-length" form before
performing the operation:
https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file#maximum-path-length-limitation
On Windows, this function returns the extended-length absolute version of path.
On other platforms it returns path unchanged.
"""
if sys.platform.startswith("win32"):
path = path.resolve()
path = Path(get_extended_length_path_str(str(path)))
return path
As there are ... many different ways subprocess.(run|call|check_output|Popen)
are invoked, this could require a rather lot of changes to handle uniformly... and would still break in a number of downstream user cases where users do not/cannot apply the registry hack, or forget to use this kind-of-arcane construction.
So ripgrep
did add a fix ( https://github.com/BurntSushi/ripgrep/commit/db6bb21a629d5b1ec1bfe89c693b280497c9eedc ) as part of the 14.0.0 release last year
Might be worth checking if issues that still see this are using a recent version of ripgrep
. If so, we may need a new issue upstream
Comes from https://github.com/conda-forge/staged-recipes/issues/17519#issue-1101991932
Actual Behavior
Prefix replacement fails with too long paths on Windows if
ripgrep
is used. The exception is not caught as it should because the subprocess is never launched:Expected Behavior
Long paths should be handled correctly or, if impossible, fallback safely to the Python implementation. Also, right now, this performance trick is either all-in or nothing. There's no logic to accept individual file errors on the ripgrep approach.
Uninstalling
ripgrep
or manglingrg
path disables this code path and makes everything work at a lower performance.Steps to Reproduce
Use this recipe.
Full CI logs: full_log.txt
Output of conda info