Extract nix data for any flake

[zz1874]

Hey, I've worked in the data extraction for any flake, and I'm wondering if we deal with a random flake, say github:rapenne-s/bento from @rapenne-s, what information do we expect?

Since it is not from nixpkgs , it doesn't have a default.nix, which prevent us from the

import (builtins.getFlake "github:rapenne-s/bento") {}

query. If we import the flake.nix file, it will only give us inputs and outputs attributes, which we can also query from the command line

$ nix flake metadata github:rapenne-s/bento
Resolved URL:  github:rapenne-s/bento
Locked URL:    github:rapenne-s/bento/f85a445532887a45f036ec61dd805cf230234d73
Description:   bento: an asynchronous NixOS deployment tool
Path:          /nix/store/zxfgh21gwxdx8gk3jwi3vd3vmznd6dv2-source
Revision:      f85a445532887a45f036ec61dd805cf230234d73
Last modified: 2023-01-07 16:18:47
Inputs:
└───nixpkgs: github:NixOS/nixpkgs/fe76645aaf2fac3baaa2813fd0089930689c53b5

$ nix flake show github:rapenne-s/bento
github:rapenne-s/bento/f85a445532887a45f036ec61dd805cf230234d73
└───packages
    └───x86_64-linux
        └───default: package 'bento'

, implying that it depends on this specific commit of nixpkgs. I don't know where to find the dependencies of this project.

[GuillaumeDesforges]

About the situation on default .nix

This happens typically with this command:

$ NIXPKGS_FLAKE_REF="github:rapenne-s/bento" nix eval --json --file "./nixpkgs-graph.nix"
error: opening file '/nix/store/zxfgh21gwxdx8gk3jwi3vd3vmznd6dv2-source/default.nix': No such file or directory

Which comes from this line: https://github.com/tweag/nixpkgs-graph-explorer/blob/3b0c8d531128f385abcd238c294ce4c0914b2714/etl/nixpkgs-graph.nix#L6

Basically, getFlake X returns an object which, when cast to a string, is the path to the realized derivation of the fetch of the flake. So import (getFlake "x") really reads import "/nix/store/aaaaaaaaaaaaaa-source/". Then by default a Nix import will look for a default.nix file. If it's not there, it fails. Hence the error.

The reason we used import was very specific to the internship from which this Nix code originated from, as we need to configure the import to allow some "insecure package". In our case, I would say that we don't need it.

What I would suggest is to stop using import and embrace flakes.

Here is a potential change:

diff --git a/etl/nixpkgs-graph.nix b/etl/nixpkgs-graph.nix
index db241c1..6ad5b9c 100644
--- a/etl/nixpkgs-graph.nix
+++ b/etl/nixpkgs-graph.nix
@@ -1,17 +1,10 @@
 # usage:
-# NIXPKGS_FLAKE_REF="github:nixos/nixpkgs/master" nix eval --json --file "./nixpkgs-graph.nix"
+# TARGET_FLAKE_REF="github:nixos/nixpkgs/master" nix eval --json --file "./nixpkgs-graph.nix"

 let
-  nixpkgsFlakeRef = builtins.getEnv "NIXPKGS_FLAKE_REF";
-  pkgs = import (builtins.getFlake nixpkgsFlakeRef) {
-    config = {
-      # package 'python-2.7.18.6' is marked as insecure
-      # we need to allow this package otherwise it is refusing to evaluate
-      permittedInsecurePackages = [
-        "python-2.7.18.6"
-      ];
-    };
-  };
+  pkgs = builtins.getFlake "nixpkgs";
+  targetFlakeRef = builtins.getEnv "TARGET_FLAKE_REF";
+  targetFlakePkgs = (builtins.getFlake targetFlakeRef).packages;
 in

 with pkgs.lib;
@@ -87,5 +80,5 @@ in

 (collect
   (x: x ? outputPath)
-  (mapAttrs (recurse "") pkgs)
+  (mapAttrs (recurse "") targetFlakePkgs)
 )

gives

$ TARGET_FLAKE_REF="github:rapenne-s/bento" nix eval --json --file "./nixpkgs-graph.nix"
[]

not quite there yet, but there is some progress.

[rapenne-s]

The notation github:rapenne-s/bento should take the default package as explained by @GuillaumeDesforges , but the notation github:rapenne-s/bento#bento would give an explicit package from a flake, both should work because they are natural when using flakes.

You can find information about flakes URL in the official documentation https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake.html#url-like-syntax

[GuillaumeDesforges]

The notation github:rapenne-s/bento should take the default package as explained by @GuillaumeDesforges , but the notation github:rapenne-s/bento#bento would give an explicit package from a flake, both should work because they are natural when using flakes.

Ah, this is not what I had in mind.

What we've been doing is scan for all available derivations (hence packages) in nixpkgs, so I was applying it there as well and looking for all packages in the flake.

So the input is really a flake, not an identifier of a flake output. See https://nixos.org/manual/nix/stable/language/builtins.html#builtins-getFlake

[rapenne-s]

Ok, indeed, I'd prefer to be specific and be able to target only a single package, if possible :)