edolstra / flake-compat

MIT License
251 stars 71 forks source link

Allow overriding the tarball fetcher #62

Open enobayram opened 12 months ago

enobayram commented 12 months ago

This change allows the caller to replace the fetchTarball used for fetching the inputs of the flake with pkgs.fetchzip, which works as a drop-in replacement. The difference is that builtins.fetchTarball does the fetching at Nix evaluation time, while pkgs.fetchzip does the fetching at build time. The distinction becomes crucial when one attempts to build a flake output inside a recursive-nix derivation. A vanilla nix build ${someFlake} or a nix-build default.nix using the current flake-compat will fail inside a recursive Nix when fetchTarball attempts to download the tarball, but the following derivation (using the flake-compat from this PR) will successfully build (on a system with recursive-nix):

let
  flake-compat = fetchTarball {
    url = "https://github.com/enobayram/flake-compat/archive/6ef9397736efb0f6075188aa3488022d1b85c11d.tar.gz";
    sha256 = "sha256:0sviihalf7lrlg4pfajywck50hi01ksax613kz6mldafb5i2g3cc";
  };
  pkgs = import <nixpkgs> {};
  buildInRec-nix = pkgs.writeText "buildInRec.nix" ''
    {flake}: let
      src = "''${flake}";
      fetchTarball = (import ${pkgs.path} {}).fetchzip;
      defaultNix = (import ${flake-compat} { inherit src fetchTarball; }).defaultNix;
      in defaultNix
  '';
  someExampleFlake = fetchTarball {
    url = "https://github.com/cachix/cachix/archive/67487d306ac5e8725e5eefa4ef50706723b986c5.tar.gz";
    sha256 = "sha256:1ks0ikha2qvr6bcypfzkgjgirvqaw1sxfzdkz9axkfy91jnp5szd";
  };
in pkgs.runCommand "recursive-flake"
  {
    requiredSystemFeatures = [ "recursive-nix" ];
    buildInputs = [ pkgs.nix ];
  }
  ''
    ln -s $(
      nix-build ${buildInRec-nix} \
        --arg flake ${someExampleFlake}
        -A packages.${builtins.currentSystem}.default
    ) > $out
  ''

In the above Nix expression, if you remove the fetchTarball argument from line 11, then the build will fail with:

       error: unable to download 'https://api.github.com/repos/NixOS/nixpkgs/tarball/970a59bd19eff3752ce552935687100c46e820a5': Couldn't resolve host name (6)

Building a flake recursively like this allows us to hide its evaluation inside a derivation, which can be a world of difference in some cases (see this discussion thread I've opened a few months ago where I've explained in detail how bad this can get).

This approach essentially allows us to get a kind of nix flake output evaluation caching as part of a larger Nix evaluation.