holaguz
commented
3 months ago Oops, wrong repo.
Opened a new issue on nixpkgs: https://github.com/NixOS/nixpkgs/issues/301376
Closed holaguz closed 1 day ago
holaguz
commented
3 months ago Oops, wrong repo.
Opened a new issue on nixpkgs: https://github.com/NixOS/nixpkgs/issues/301376
holaguz
commented
3 months ago Reopening this after the following comment: https://github.com/NixOS/nixpkgs/issues/301376#issuecomment-2038332592
pho
commented
3 months ago I'm also I'm triggering the same error on Arch & nix 2.21.1:
$ nix repl --expr 'builtins.fetchTarball { url = "http://more.musl.cc/11.2.1/x86_64-linux-musl/x86_64-linux-musl-cross.tgz"; }'
Nix 2.21.1
Type :? for help.
error:
… while calling the 'fetchTarball' builtin
at «string»:1:1:
1| builtins.fetchTarball { url = "http://more.musl.cc/11.2.1/x86_64-linux-musl/x86_64-linux-musl-cross.tgz"; }
| ^
error: file 'x86_64-linux-musl-cross/bin/x86_64-linux-musl-gcc' in tarball has unsupported file typeLooks like the files which fetchTarball complains about are hardlinks, which (TIL) tar knows how to preserve
Repacking the tgz files with tar --hard-dereference solves this, so my workaround has been to repack it, serve it locally and fetch it.
EDIT: I just updated to nix 2.21.2, and its still happening
pho
commented
3 months ago In case it's any help, I have git-bisected this and looks like https://github.com/NixOS/nix/commit/cabee9815239af426cece729cb765810b8a716ce might be the culprit
holaguz
commented
3 months ago The error is an exception being thrown on this file: https://github.com/NixOS/nix/blob/3fd8dfec4d0747e8f1129dc7f43c1f89fe3ba432/src/libutil/tarfile.cc#L223
Meaning the file is neither AE_IFDIR, AE_IFREG, or AE_IFLNK. For reference, these are the filetypes defined for libarchive:
man 3 archive_entry_stat:
General accessor functions
The functions archive_entry_filetype() and archive_entry_set_filetype() get respectively set the
filetype. The file type is one of the following constants:
AE_IFREG Regular file
AE_IFLNK Symbolic link
AE_IFSOCK Socket
AE_IFCHR Character device
AE_IFBLK Block device
AE_IFDIR Directory
AE_IFIFO Named pipe (fifo)That's as far as I could dig since I'm not familiar with the source code nor libarchive.
Looks like the files which fetchTarball complains about are hardlinks, which (TIL) tar knows how to preserve
In my case bin/sparc-gaisler-elf-gcc is indeed a hardlink to bin/sparc-gaisler-elf-gcc-10.2.0
thufschmitt
commented
3 months ago Per https://github.com/libarchive/libarchive/blob/341800da257fb8c31b8b4281604fb7e3436b61f3/libarchive/archive_read_support_format_tar.c#L1219 , it seems like libarchive doesn't really use these accessor functions when unpacking tarfiles, but reads the tartype instead (both seem to be in sync, except for hardlinks which aren't handled by the AE_IF* family and for which archive_entry_filetype returns 0).
Maybe unpackTarfileToSink should follow that behavior.
For a more self-contained reproducer:
$ mkdir archive
$ touch archive/foo
$ ln archive/foo archive/bar
$ tar cf archive.tar archive
$ nix-instantiate --eval --expr 'builtins.fetchTarball "file://'$PWD'/archive.tar"'
error:
… while calling the 'fetchTarball' builtin
at «string»:1:1:
1| builtins.fetchTarball "file:///home/thufschmitt/Repos/nixos/nix/work/archive.tar"
| ^
error: file 'archive/bar' in tarball has unsupported file typeMaybe unpackTarfileToSink should follow that behavior.
Contributions for that welcome ;)
infinisil
commented
1 month ago Since recently, Nixpkgs tarballs include hardlinks, which now causes issues because of this bug: https://github.com/NixOS/infra/issues/438
nixos-discourse
commented
4 weeks ago This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/2024-06-10-nix-team-meeting-minutes-151/46956/1
Describe the bug
A custom derivation using
fetchTarballsuddenly stopped building.Full error log
``` evaluating file 'When importing the nix file using the repl, the error seems to appear after downloading a few MBs:
I was able to reproduce the error on both 23.11 and unstable channels, in two separate pcs:
PC 1 (Arch 6.6.23 LTS):
Linux hera 6.6.23-1-lts #1 SMP PREEMPT_DYNAMIC Wed, 27 Mar 2024 07:47:20 +0000 x86_64 GNU/Linuxnix-env (Nix) 2.21.1PC 2 (Arch 6.8.2):
Linux artemis 6.8.2-arch2-1 #1 SMP PREEMPT_DYNAMIC Thu, 28 Mar 2024 17:06:35 +0000 x86_64 GNU/Linuxnix-env (Nix) 2.21.1I couldn't reproduce it on the following computer:
Linux TITAN 6.5.0-26-generic #26~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Mar 12 10:22:43 UTC 2 x86_64 x86_64 x86_64 GNU/Linuxnix-env (Nix) 2.19.2You can find the MRE down below:
toolchain.nix
```nix { pkgs }: let version = "2.2.1"; hash = "097z15zshr4711rzfr8ccwl8vzpxp7da573p89qmngxzrwxr4kwl"; tarballUrl = "https://www.gaisler.com/anonftp/bcc2/bin/bcc-${version}-gcc-linux64.tar.xz"; bccToolchain = pkgs.stdenv.mkDerivation { name = "gaisler-bcc-${version}"; src = builtins.fetchTarball { url = tarballUrl; sha256 = hash; }; }; in { inherit bccToolchain; } ```Feel free to ping me if I missed any details.