fetchcargo is non-deterministic

abbradar commented 8 years ago

Basic info

To make sure that we are on the same page:

Kernel: (run uname -a) 4.5.0
System: (NixOS: nixos-version, Ubuntu/Fedora: lsb_release -a, ...) 16.09.git.cd86f80 (Flounder)
Nix version: (run nix-env --version) 1.11.2
Nixpkgs version: (run nix-instantiate --eval '<nixpkgs>' -A lib.nixpkgsVersion) 16.09.git.cd86f80
Describe your issue here

During testing of https://github.com/NixOS/nixpkgs/pull/14088 I managed to get at least two different hashes for exa. After some discussion with @jgillich it seems that this derivation can produce non-deterministic output.

Expected result

Hashes of fetchcargo outputs are always the same.

Actual result

They are not.

Steps to reproduce

Build exa several times.

despairblue commented 8 years ago

How would one work around that? It basically means the system cannot be rebuild :cry:

abbradar commented 8 years ago

cc @wizeman

abbradar commented 8 years ago

@despairblue You can work around it by updating SHA checksum in the affected package locally.

despairblue commented 8 years ago

@abbradar :confused: Is there any documentation you could point me too. I know how I would to it for packages that I install as a user:

checking out nixpkgs repo
find the last commit that build for unstable on hydra
check that out
change the exa's default.nix file 5 run nix-env -f ~/git/nixpkgs -i exa

But I think it's also possible to override it in a nix expression, like the one in /etc/nixos/configuration.nix, since I installed it as a systems package. The thing is that all ressources I find either explain how to do it for .nixpkgs/config.nix or are that much out of date that they link to non existent documentation

despairblue commented 8 years ago

I changed my configuration.nix to include that:

  environment.systemPackages = with pkgs; [
    # ...
    (pkgs.lib.overrideDerivation pkgs.exa (attrs: {
      depsSha256 = "1klg78ynqvj5gbs4cqsah4vmdfv8iv2va4fsv41l3g0ssja6z5xm";
    }))
    # ...
  ];

but to no avail :confused:

abbradar commented 8 years ago

Sadly we don't have a mechanism to override depsSha256 (it needs to be supported). However, you can rebuild your whole NixOS from a modified repository:

$ nixos-rebuild switch -I nixpkgs=/absolute/path/to/patched/nixpkgs

abbradar commented 8 years ago

Another way to do it is to call patched nixpkgs from your configuration.nix:

(import /path/to/nixpkgs {}).exa

despairblue commented 8 years ago

Thanks for your help! I really appreciate it.

I tried your last suggestions, but that leads to:

building path(s) ‘/nix/store/2w7vws133m7x7ib31js9ric1gpq5wys7-exa-2016-04-20’
unpacking sources
unpacking source archive /nix/store/884pf1gxkcxiydpkfcjfrwvy3jsxax1l-exa-110a1c716bfc4a7f74f74b3c4f0a881c773fcd06-src
source root is exa-110a1c716bfc4a7f74f74b3c4f0a881c773fcd06-src
Using cargo deps from /nix/store/7h2gz0ghd9xsqjymhp22q893ync0zzaa-exa-2016-04-20-fetch
Using indexHash '-ba82b75dd6681d6f'
Using rust registry from /nix/store/wcl2cfdl2v1hfgmgklqwdh71gpb62dwf-rustRegistry-2016-08-10-fe018be
warning: custom registry support via the `registry.index` configuration is being removed, this functionality will not work in the future
    Updating git repository `https://github.com/rust-datetime/zoneinfo-compiled.git`
warning: spurious network error (2 tries remaining): [12/-1] curl error: Couldn't resolve host 'github.com'

warning: spurious network error (1 tries remaining): [12/-1] curl error: Couldn't resolve host 'github.com'

error: failed to load source for a dependency on `zoneinfo_compiled`

Caused by:
  Unable to update https://github.com/rust-datetime/zoneinfo-compiled.git#f56921ea

Caused by:
  failed to fetch into /tmp/nix-build-exa-2016-04-20.drv-0/deps/git/db/zoneinfo-compiled-4887a505c85fb388

To learn more, run the command again with --verbose.
builder for ‘/nix/store/zs8481q35d8xl5gjqvw4v6zv61qgx2rg-exa-2016-04-20.drv’ failed with exit code 1
cannot build derivation ‘/nix/store/l6k5mknhb3hn7g5k43q5iwv471ndbapn-system-path.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/4l9cc5l452h5p2mqz56ck96sq4jn86az-nixos-system-firefly-16.09pre88945.5120af0.drv’: 1 dependencies couldn't be built
error: build of ‘/nix/store/4l9cc5l452h5p2mqz56ck96sq4jn86az-nixos-system-firefly-16.09pre88945.5120af0.drv’ failed

It looks like some network hick up, but I tried it multiple times and I opened the github repository from the same machine and it loads fine.

Maybe it has something to do with this line: warning: custom registry support via theregistry.indexconfiguration is being removed, this functionality will not work in the future ?

abbradar commented 8 years ago

Hm, unfortunately I have no idea what's going on -- you've fixed the hash, so it seems like another problem in fetchcargo.

despairblue commented 8 years ago

Still thanks for the help. I learned a lot!

globin commented 8 years ago

exa depends on a git repository in Cargo.toml which doesn't work with fetchcargo

Mic92 commented 8 years ago

maybe fetchcargo could take an argument to pin git-based dependencies on a certain commit?

ticki commented 7 years ago

I can confirm that this is still an issue when building exa:


error: failed to load source for a dependency on `zoneinfo_compiled`

Caused by:
  Unable to update https://github.com/rust-datetime/zoneinfo-compiled.git#f56921ea

through AUR/exa-git.

cc @ogham

mmahut commented 5 years ago

Are there any updates on this issue, please?

Mic92 commented 5 years ago

Should be fixed.

NixOS / nixpkgs