The filename is too long

[benoit-cty]

Hello @conda-forge/staged-recipes ,

My recipe failed because of long pathnames in source:

FileNotFoundError: [WinError 206] The filename or extension is too long

Azure CI Win64 Source

We did use long pathname: we have a many subfolders to describe French Taxes. We limit them to 195 characters max (including source folder name), as Windows have a default limit of 256 characters.

The Windows CI failed because of that : We need to reduce length by 30 characters to make it work in staged-recipes CI. That's too much to keep understandable path.

On my Windows the package build well with conda build --croot d:\tmp .conda to shorten the build folder.

Do you think it's acceptable to do the same tricks in staged-recipes ? It seem that it need only a modification of https://github.com/conda-forge/staged-recipes/blob/eacdb0130d5174d5dd5bcd4c33b86ebc542996a4/.ci_support/build_all.py#L126

Are we allowed to have a specific script for our feedstock ?

[jakirkham]

We already set the build path to a short path in the feedstocks.

Not sure why that wouldn't be the case here as well. @conda-forge/core, does anyone know?

[benoit-cty]

Thanks, I tried to add CONDA_BLD_PATH in my recipe: https://github.com/openfisca/staged-recipes/blob/d0410e41c4b98c181a12d2b69fbc1970a2085658/.azure-pipelines/azure-pipelines-win.yml#L14

And here is the corresponding PR: https://github.com/conda-forge/staged-recipes/pull/17483

but it does not work: index_path= C:\Miniconda\conda-bld recipes_dir= D:\a\1\s\recipes folders= ['openfisca-france'] See https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=441065&view=logs&j=240f1fee-52bc-5498-a14a-8361bde76ba0&t=06d62fd2-ab49-5add-9ee5-83b6882d7d2b&l=532

[jaimergp]

Building did work, it's the hardcoded prefix detection with ripgrep that fails :/ Maybe that's the one causing issues? @benoit-cty do you have ripgrep installed in your local system? I think conda will default to another utility if ripgrep is not present.

[benoit-cty]

Thanks a lot ! When adding conda remove -y ripgrep in https://github.com/conda-forge/staged-recipes/pull/17483 the CI run fine: https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=441126&view=logs&j=240f1fee-52bc-5498-a14a-8361bde76ba0&t=06d62fd2-ab49-5add-9ee5-83b6882d7d2b&l=342

But I suppose you could not merge and updated CI this way, especially if it slow down every build ?

[jaimergp]

Instead of modifying the workflow files (which can't be edited by submissions), I would rename the rg executable (I think it's called like that for ripgrep) in your build.sh or equivalent. It should have the same effect and it's a workaround the @conda-forge/core team would be more comfortable with. Let us know!

[benoit-cty]

I did not manage to do it, I tried move %PREFIX%\Library\bin\rg.exe %PREFIX%\Library\bin\rg-desactivate.exe without success : https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=442087&view=logs&j=240f1fee-52bc-5498-a14a-8361bde76ba0&t=06d62fd2-ab49-5add-9ee5-83b6882d7d2b&l=1637 Did you know where rg.exe is ?

[jaimergp]

It might in the conda-build environment (not your package's prefix used during conda build). See here. That means it must be under C:\Miniconda\Library\bin\rg.exe.

[benoit-cty]

Thank you very much, it works !

[jakirkham]

This looks potentially related ( https://github.com/BurntSushi/ripgrep/issues/364 )

[jaimergp]

Oh nice find! conda-build should also fallback automatically to the non-ripgrep option when ripgrep fails. This needs to be fixed in the conda-build logic. Right now:

• it looks like it's either "all ripgrep" or "no ripgrep", which wouldn't allow for individual errors on specific paths.
• it needs to catch FileNotFoundError too

That said, it doesn't look like the process is ever started, so ripgrep is not being used yet. It's the subprocess launch that fails:

  File "C:\Miniconda\lib\site-packages\conda_build\build.py", line 2366, in build
    newly_built_packages = bundlers[pkg_type](output_d, m, env, stats)
  File "C:\Miniconda\lib\site-packages\conda_build\build.py", line 1672, in bundle_conda
    output['checksums'] = create_info_files(metadata, replacements, files, prefix=metadata.config.host_prefix)
  File "C:\Miniconda\lib\site-packages\conda_build\build.py", line 1285, in create_info_files
    files_with_prefix = get_files_with_prefix(m, replacements, files, prefix)
  File "C:\Miniconda\lib\site-packages\conda_build\build.py", line 949, in get_files_with_prefix
    pfx_matches = have_regex_files([f[2] for f in files_with_prefix], prefix=prefix,
  File "C:\Miniconda\lib\site-packages\conda_build\build.py", line 526, in have_regex_files
    match_records_rg = regex_files_rg(files, prefix, tag,
  File "C:\Miniconda\lib\site-packages\conda_build\build.py", line 324, in regex_files_rg
    raise e
  File "C:\Miniconda\lib\site-packages\conda_build\build.py", line 317, in regex_files_rg
    matches = subprocess.check_output(args, shell=False).rstrip(b'\n').split(b'\n')
  File "C:\Miniconda\lib\subprocess.py", line 424, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "C:\Miniconda\lib\subprocess.py", line 505, in run
    with Popen(*popenargs, **kwargs) as process:
  File "C:\Miniconda\lib\subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Miniconda\lib\subprocess.py", line 1420, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 206] The filename or extension is too long

This is somewhat surprising because most prefixes should exceed the 260 limit anyway due to the placeholders in PREFIX. This is worth an issue in conda-build. I am guessing Azure has enabled long paths by default?

[MementoRC]

Just in case this helps. I had a similar issue that: git config --system core.longpaths true solved. maturin failed to download one of the dependencies with git clone