Open wizeman opened 8 years ago
FWIW, I've just encountered a similar error (I'm also using nixops but am running on the public hydra).
fetching path ‘/nix/store/dwcx9w994i9kl19gfvcmfi8xz7f2nsjf-stage-2-init.sh’...
*** Downloading ‘http://hydra.nixos.org/nar/dwcx9w994i9kl19gfvcmfi8xz7f2nsjf-stage-2-init.sh’ (signed by ‘hydra.nixos.org-1’) to ‘/nix/store/dwcx9w994i9kl19gfvcmfi8xz7f2nsjf-stage-2-init.sh’...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (22) The requested URL returned error: 410 Gone
bzip2: Compressed file ends unexpectedly;
perhaps it is corrupted? *Possible* reason follows.
bzip2: Inappropriate ioctl for device
Input file = (stdin), output file = (stdout)
It is possible that the compressed file(s) have become corrupted.
You can use the -tvv option to test integrity of such files.
You can use the `bzip2recover' program to attempt to recover
data from undamaged sections of corrupted files.
error: unexpected end-of-file
download of ‘http://hydra.nixos.org/nar/dwcx9w994i9kl19gfvcmfi8xz7f2nsjf-stage-2-init.sh’ failed: No such file or directory
could not download ‘/nix/store/dwcx9w994i9kl19gfvcmfi8xz7f2nsjf-stage-2-init.sh’ from any binary cache
fetching path ‘/nix/store/dwcx9w994i9kl19gfvcmfi8xz7f2nsjf-stage-2-init.sh’ failed with exit code 1
fetching path ‘/nix/store/ff7nk1awqi1wgwmbnbjmn7g5k4srdx67-errors-2.0.1.tar.gz’...
killing process 4145
error: some substitutes for the outputs of derivation ‘/nix/store/qhflwr9zyyilybldjbbdf3a9af0jycxi-stage-2-init.sh.drv’ failed (usually happens due to networking issues); try ‘--fallback’ to build derivation from source
error: unable to build all machine configurations
This was after performing a local nix-collect-garbage.
I've not had a chance to investigate much yet.
Apparently, this also happens to me with nixos-rebuild
(i.e., even when nixops
is not involved).
And even worse, the --fallback
option for nixos-rebuild
doesn't work:
$ nixos-rebuild switch --fallback
building Nix...
building the system configuration...
fetching path ‘/nix/store/n2rhmzj3hy7b2gg1ba4dipsrqxrxwixy-rg-src’...
*** Downloading ‘http://homeserv:3000/nar/n2rhmzj3hy7b2gg1ba4dipsrqxrxwixy-rg-src’ to ‘/nix/store/n2rhmzj3hy7b2gg1ba4dipsrqxrxwixy-rg-src’...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (22) The requested URL returned error: 410 Gone
bzip2: Compressed file ends unexpectedly;
perhaps it is corrupted? *Possible* reason follows.
bzip2: Inappropriate ioctl for device
Input file = (stdin), output file = (stdout)
It is possible that the compressed file(s) have become corrupted.
You can use the -tvv option to test integrity of such files.
You can use the `bzip2recover' program to attempt to recover
data from undamaged sections of corrupted files.
error: unexpected end-of-file
download of ‘http://homeserv:3000/nar/n2rhmzj3hy7b2gg1ba4dipsrqxrxwixy-rg-src’ failed: No such file or directory
could not download ‘/nix/store/n2rhmzj3hy7b2gg1ba4dipsrqxrxwixy-rg-src’ from any binary cache
fetching path ‘/nix/store/n2rhmzj3hy7b2gg1ba4dipsrqxrxwixy-rg-src’ failed with exit code 1
error: some substitutes for the outputs of derivation ‘/nix/store/c7sm6c47l9s6aijimsnxnkl2sv0gk5jj-rg-src.drv’ failed (usually happens due to networking issues); try ‘-
-fallback’ to build derivation from source
(use ‘--show-trace’ to show detailed location information)
I will file a separate bug for --fallback
not working.
i think the best option is to use a fetchurl call, so it can reproduce the output from any system after a GC
In my case, it seems that the derivation which fails to download (because it got wrongly garbage collected) is the output of a pkgs.srcOnly
derivation (the derivation with name kodiOff-src
), which is itself an input to the import
command in the kodifOff_pkg
variable.
So if I were to guess, I would say that in my case, the problem comes from doing an import
from a derivation output, which seems to be breaking the chain of references when doing the garbage collection (i.e., the GC doesn't consider the input to the import
as being needed, so it deletes it, even though it is obviously needed).
Relevant to #954
This bug prevents the entire import-from-derivation feature from working reliably in the presence of garbage collection, so IMHO this is a serious issue.
This is really a bother. I have a lot of things in my system configuration that required import-from-derivation (IFD), so whenever I run a GC and try to redeploy, it first needs to download and build all kinds of things for ~5minutes.
I also use IFD a lot in my config files, and this issue is quite annoying for me. As a workaround, I added fake runtime dependencies with writeText for the derivations that are then subsequently imported, and this prevents nix-collect-garbage from gc'ing the imports
For things downloaded with the builtin fetchers (builtins.fetchTarball
and builtins.fetchurl
should work too) I just found this workaround: By using indirect GC roots you can gcroot the whole ~/.cache/nix/tarballs directory:
find ~/.cache/nix/tarballs -type l -execdir ln -svf ~/.cache/nix/tarballs/{} /nix/var/nix/gcroots/auto/{} \;
Or a bit simpler but not as clean (links everything (harmless) and fails if there's nothing to link):
ln -sfv ~/.cache/nix/tarballs/* /nix/var/nix/gcroots/auto
Another occurrence is:
/nix/store/vihay323q5fnimdm6gmfc50slngjdpcg-ghc-8.8.3/bin/ghc-pkg: line 12: /nix/store/vihay323q5fnimdm6gmfc50slngjdpcg-ghc-8.8.3/lib/ghc-8.8.3/bin/ghc-pkg: No such file or directory
@edolstra would it makes sense for Nix to add all derivations during build time to a temporary GC root?
I marked this as stale due to inactivity. → More info
I still care about this
Any movement?
See also https://github.com/NixOS/nix/pull/3494 for a PR involving logging IFD.
I marked this as stale due to inactivity. → More info
Still important
I marked this as stale due to inactivity. → More info
spam
I have a NixOS configuration with the following contents:
Now, even though I have the following options in the
/etc/nix/nix.conf
of my private Hydra server:... and even though the configuration root hasn't been gc'ed, a Nix garbage collection still appears to delete the output of the
pkgs.srcOnly
derivation.The above configuration is built by my private Hydra server, but if a garbage collection runs after the configuration is built, then my laptop (which is configured to use my Hydra server as a binary cache) fails to download the package during
nixops deploy
, reporting a410 Gone
error instead:This happens to me all the time.
If I add the
--fallback
option, thenixops deploy
command successfully completes (and most other packages are correctly downloaded instead of being rebuilt), but then I can't be sure that there won't be spurious rebuilds (say, because my Hydra server hasn't finished rebuilding packages).But in any case, correct me if I'm wrong, but I think that according to the documentation, the garbage collection was not supposed to delete that derivation.