Symlinks fail to extract in tar

jeroen commented 6 years ago

Consider the following example:

curl -OL https://cran.r-project.org/src/base/R-3/R-3.4.4.tar.gz
tar -xf R-3.4.4.tar.gz

This command fails in msys2. The reason is that the archive contains symlinks, but within the archive the symlink appears before the actual file that it links to. Therefore tar fails to resolve the link on-the-fly because it can't link to something that isn't there yet.

The issue here is that the R-3.4.4.tar.gz file was created on macOS (with bsdtar) which does not have a --sort option. Hence random ordering of files within a tar is not illegal and very common.

A workaround is the following:

set MSYS=winsymlinks:lnk
tar -xf R-3.4.4.tar.gz

This will replace symlinks in the tar archive by windows shortcuts instead of trying to resolving them on the fly.

It took me two days to find this solution. Perhaps MSYS=winsymlinks:lnk should be the default in tar like it is in Cygwin? Or alternatively tar should be modified to first extract real files before extracting symlinks.

jeroen commented 6 years ago

Interestingly this issue seems the opposite of https://github.com/Alexpux/MSYS2-packages/issues/140. Has the default value for winsymlinks changed in the mean while?

mingwandroid commented 6 years ago

The default should be whatever works best on the given system (do when supported, native symlinks).

lnk files go not work with the standard C library functions on Windows unfortunately. The are interpreted 'correctly' by Cygwin and Windows explorer only.

angelog0 commented 6 years ago

@jeroen wrote:

A workaround is the following:

set MSYS=winsymlinks:lnk tar -xf R-3.4.4.tar.gz

..another workaround is to run tar two times:

tar -xf R-3.4.4.tar.gz    # Now it prints errors...
tar -xf R-3.4.4.tar.gz    # Now all is fine...

alexjorgef commented 4 years ago

@jeroen wrote:

A workaround is the following: set MSYS=winsymlinks:lnk tar -xf R-3.4.4.tar.gz

..another workaround is to run tar two times:
tar -xf R-3.4.4.tar.gz    # Now it prints errors...
tar -xf R-3.4.4.tar.gz    # Now all is fine...

I think I'm experiencing something similar:

tar: ansible-2.9.7/lib/ansible/plugins/lookup/_openshift.py: Cannot create symlink to ‘k8s.py’: No such file or directory
tar: ansible-2.9.7/test/integration/targets/supervisorctl/tasks/install_Darwin.yml: Cannot create symlink to ‘install_pip.yml’: No such file or directory
...
tar: Exiting with failure status due to previous errors
==> ERROR: A failure occurred in prepare().

~~The next solution work for me~~, PKGBUILD prepare function:

...
prepare() {
  [[ -d ${pkgname}-${pkgver} ]] && rm -rf ${pkgname}-${pkgver}
  tar zxf "${srcdir}/${pkgname}-${pkgver}.tar.gz" || true
  tar zxf "${srcdir}/${pkgname}-${pkgver}.tar.gz"
}
...

And this errors occur on final steps of build:

...
warning: BuildScriptsCommand: bin/ansible-playbook is an empty file (skipping)
warning: BuildScriptsCommand: bin/ansible-pull is an empty file (skipping)
warning: BuildScriptsCommand: bin/ansible-doc is an empty file (skipping)
warning: BuildScriptsCommand: bin/ansible-galaxy is an empty file (skipping)
warning: BuildScriptsCommand: bin/ansible-console is an empty file (skipping)
copying and adjusting bin/ansible-connection -> build/scripts-3.8
warning: BuildScriptsCommand: bin/ansible-vault is an empty file (skipping)
warning: BuildScriptsCommand: bin/ansible-config is an empty file (skipping)
warning: BuildScriptsCommand: bin/ansible-inventory is an empty file (skipping)
copying and adjusting bin/ansible-test -> build/scripts-3.8
error: [Errno 2] No such file or directory: 'build/scripts-3.8/ansible-playbook'

Edit: After some builds, realize that BuildScriptsCommand need symlinks as shortcuts:

...
prepare() {
  [[ -d ${pkgname}-${pkgver} ]] && rm -rf ${pkgname}-${pkgver}
  tar zxf "${srcdir}/${pkgname}-${pkgver}.tar.gz" || true
  MSYS=winsymlinks:lnk tar zxf "${srcdir}/${pkgname}-${pkgver}.tar.gz"
}
...