search

conda-forge / conda-smithy

The tool for managing conda-forge feedstocks.
https://conda-forge.org/
BSD 3-Clause "New" or "Revised" License
147 stars 176 forks source link

AppVeyor/Windows: Extract Symbolic Links #1002

Open ax3l opened 5 years ago

ax3l commented 5 years ago

Since yesterday, the AppVeyor/Windows builds seem to fail when trying to extract a source code repo that contains symbolic links:

https://ci.appveyor.com/project/conda-forge/openpmd-api-feedstock/builds/21656144/job/42yuuwju3tuyexcc

https://github.com/openPMD/openPMD-api/tree/0.7.0-alpha/docs/source/dev

In the above case, the links are not needed for the build, but it does not even get there anymore ;)

effigies commented 5 years ago

I can replicate this issue in conda-forge/nibabel-feedstock#14.

isuruf commented 5 years ago

Can you open this issue against conda-build?

grlee77 commented 5 years ago

I have seen this issue on windows for both conda-build 3.16.3 and 3.17.5. It did not occur for 3.15.4.

A quick look over at conda-build seems to indicate the following libarchive PR as the one that changed this behavior: conda/conda-build#3163

isuruf commented 5 years ago

cc @mingwandroid

effigies commented 5 years ago

@isuruf Opened conda/conda-build#3351.

mingwandroid commented 5 years ago

Hey.

We enable 'developer mode' in our internal CI which is the correct thing to do.

Our team will not have time to look into this unfortunately. The options are:

  1. Ask AppVeyor if they can enable developer mode.
  2. Move to Azure if it allows / enables developer mode.
  3. Implement a fallback in conda-build so that should unpacking fail with a symlink related permissions error, the python tarfile module gets used instead.
  4. Implement a fallback in libarchive so that should unpacking fail with a symlink related permissions error, the C code therein falls back to copying instead. This is trickier than it would initially appear because at the point when a symlink is created it is not necessarily the case that the target of the symlink exists due to the ordering of the tarfile, so you may not have any file to copy. This gets worse with symlinks to symlinks to files so must be handled carefully as a post-process. This would be a continuation of my PR at https://github.com/libarchive/libarchive/pull/1030

Cheers!