NixOS / nix

Nix, the purely functional package manager
https://nixos.org/
GNU Lesser General Public License v2.1
12.12k stars 1.47k forks source link

How to get the build-time dependencies of a package? #1245

Open copumpkin opened 7 years ago

copumpkin commented 7 years ago

If I have:

{
  exportReferencesGraph = [ "foo" package ];
}

I get the runtime dependencies of package. If instead I:

{
  exportReferencesGraph = [ "foo" package.drvPath ];
}

I get all the recursive build-time dependencies of package. What I actually want is the closure of the runtime dependencies of package's build-time dependencies, but I don't know how to get that.

E.g., if package is hello and building it depends on stdenv, I want the runtime closure of stdenv, not its build-time closure, since the latter will include everything all the way back to bootstrap-tools and I get a ton of stuff I don't need. When Nix builds package, it doesn't pull in all the transitive build junk.

How can I get the behavior I want?

cc @edolstra @shlevy @7c6f434c @cleverca22

Edit: this seems like the logic I want: https://github.com/NixOS/nix/blob/master/src/libstore/build.cc#L1251-L1266

domenkozar commented 7 years ago

Can't you just use package.buildInputs and iterate over those and get runtime deps?

copumpkin commented 7 years ago

That would work if only buildInputs were everything. But someone can also get build-time inputs by adding something like prePatch = "${pkgs.foo}/bin/foo bar baz" and that won't be a buildInput but will be a build input :( On Thu, Feb 23, 2017 at 03:54 Domen Kožar notifications@github.com wrote:

Can't you just use package.buildInputs and iterate over those and get runtime deps?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/NixOS/nix/issues/1245#issuecomment-281932744, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAKPxnMt3vQto_y_CPGVABq0a2E_OOHks5rfUlUgaJpZM4MIGK5 .

domenkozar commented 7 years ago

Then actually you just need to filter run-time dependency graph of package and remove package itself?

copumpkin commented 7 years ago

Not sure I understand. The runtime closure of package is too small, and the runtime closure of package.drvPath is too big, because it includes the build-time dependencies all the way down the chain. I don't know if it's possible for me to scan the closure dump from package.drvPath to figure out which of those are the runtime dependencies of my immediate build-time dependencies, either.

shlevy commented 7 years ago

This isn't currently possible, but would be if string context were introspectible (e.g. for #1052 ) or you could use recursive nix to do nix-store -q --references.

copumpkin commented 7 years ago

Well, that's disappointing! Not even possible with horrifying hackery? It kind of stifles (or significantly complicates) my dreams of a better nixos-install, at least if we want to ensure build dependencies are pre-installed on the target machine.

shlevy commented 7 years ago

You could write some C++ machinery that links to the libstore derivation parsing and read the references out of thedrv directly

copumpkin commented 7 years ago

Would that be thwarted by paths visible to a sandboxed builder?

shlevy commented 7 years ago

I'm not completely sure what you mean, but there's no way to ensure that all of the build time dependencies are live for a drv file that's referenced by a separate drv. You can ensure all the run time dependencies are live by taking a reference to the drv's outputs, of course.

copumpkin commented 7 years ago

I guess another option would be to add some sort of exportBuildReferencesGraph feature that replicates the builder resolution logic and makes sure it's available to the current builder. The other two solutions you proposed are appealing, but meatier and probably tougher to get reviewed/merged.

shlevy commented 7 years ago

Oh, it would be pretty trivial if we were to add a new feature, but also kind of annoying given that in principle all this info is available at eval time

copumpkin commented 7 years ago

Agreed 😄

7c6f434c commented 7 years ago

Well, you could iterate over derivation attributes, checking the type (collect derivations, recurse on lists and other attrsets, discard the rest)…

copumpkin commented 7 years ago

How about string dependencies though? I think that's what @shlevy was talking about On Sat, Feb 25, 2017 at 10:16 Michael Raskin notifications@github.com wrote:

Well, you could iterate over derivation attributes, checking the type (collect derivations, recurse on lists and other attrsets, discard the rest)…

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/NixOS/nix/issues/1245#issuecomment-282490255, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAKP0h2o2PwC63ZjnmNm7ARNv9Rv5I0ks5rgEXbgaJpZM4MIGK5 .

7c6f434c commented 7 years ago

I had a false hope that the derivation primop stashes context somewhere

7c6f434c commented 7 years ago

What we actually need is access to PathSet context (list of paths relevant for a string) from Nix language proper…

shlevy commented 7 years ago

@7c6f434c Yes, that's what I said above in reference to #1052

copumpkin commented 7 years ago

@shlevy @7c6f434c how silly is this? I realize I can't do it in pure Nix land, but this should get me over the hump for nixos-install.sh for now, right?

$ nix-instantiate -E "with import $PWD/nixos {}; system"
warning: you did not specify ‘--add-root’; the result might be removed by the garbage collector
/nix/store/wxgn1kiqnznx45a458fz8wgpr2vpddz1-nixos-system-nixos-17.03.git.e798f57.drv

$ nix-store -q --references $(nix-instantiate -E "with import $PWD/nixos {}; system")
warning: you did not specify ‘--add-root’; the result might be removed by the garbage collector
/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh
/nix/store/jmxir09c1x5mqi9c0svgqkn1yqmzvbki-bash-4.4-p5.drv
/nix/store/0gnrn3dbbmq0w77s1waz1xndcaw3qshi-bash-4.4-p5.drv
/nix/store/78qxjyzwrx6prqwa7aadqzjdhvyqx27p-linux-pam-1.2.1.drv
/nix/store/58i2sk7vj94j4lmvkncw25h5qiwryy42-glibc-2.25.drv
/nix/store/alhz2m83skrll1gp1pbdssjkkafhvx5a-coreutils-8.26.drv
/nix/store/awq3wvjbygz7kb7zaq0jnmz0lv3fg2g6-perl-5.22.2.drv
/nix/store/nbabgwrb2lkss4i29vqxgjn9gi0s6r7d-libcap-2.25.drv
/nix/store/wc1cy0d2w9y13r8abjc80fzmfckbvm00-kmod-23.drv
/nix/store/dgvd8ijz2ck5l6g4lc2chkgqpap4jcqw-systemd-232.drv
/nix/store/12g05f177c3qmilshdf6hl175bvmg437-polkit-0.113.drv
/nix/store/37fvvmpr7a45lsk1sqkpg432wkfjvl0w-net-tools-1.60_p20120127084908.drv
/nix/store/531giy6mqh207cadkffsi1z960l4jywz-update-users-groups.pl
/nix/store/mjhz62zab7ss9p7c2d0z24rf8pwcp605-gnugrep-3.0.drv
/nix/store/pjcfx303zgc0f5h0gbp1faa6i8knzg9l-findutils-4.6.0.drv
/nix/store/h3krnnmz1wncd2bjs2jja743294kp9cd-stdenv.drv
/nix/store/5mz7hr9wfizcjpjf87cr3nhhdqks8s14-mounts.sh.drv
/nix/store/719xq19s5nx290nl6mjhhjx05icp1mnz-dbus-1.10.16.drv
/nix/store/77m9gzwwi2lxf4vyam6h0fv98aff32xk-etc.drv
/nix/store/8prs9qwm3admvk9r6ydz8ri8gbr9xlbi-shadow-4.4.drv
/nix/store/nhykdi1pj2f0qs65dhhjp2d78c4bsal5-util-linux-2.29.drv
/nix/store/v8hqxf4gnk65j1k77ryka19y0hjf3h19-openresolv-3.8.1.drv
/nix/store/7jyhsv5fvkqy63hdf7iy43d2bcj3n3ya-stage-2-init.sh.drv
/nix/store/88k01lciiq4implclzbdn173dl4ryppv-iputils-20161105.drv
/nix/store/d40awfkvwww06ysyd5kzqrjlfw481xqm-initrd.drv
/nix/store/fy26730mgf9zg8wy7xzgcnrh2dpbip4w-linux-4.9.12.drv
/nix/store/k0aqai2r9a72r06i17hbhn3r7s85vnf7-users-groups.json.drv
/nix/store/kg35qfk83qd0v0bqn5rxalfkaff30k5x-e2fsprogs-1.43.4.drv
/nix/store/knyp6298babzqxvhhsz0p3wkvji6lkdr-sudo-1.8.19p2.drv
/nix/store/mpzawy6scqplgxzrcv25rsqw3k877qpi-kernel-modules.drv
/nix/store/mx0v33rhhds4ylq2cdklam92d7d8i3h8-security-wrapper.drv
/nix/store/wxr0wb97ixl350dvv9g7pqky68gk5zlb-fuse-2.9.7.drv
/nix/store/qw9wb81fvwrzsqdwiykpll0w8871b43z-system-path.drv
/nix/store/r620i2hsldg2ssr3nijdimyw1wyl2wlp-setup-etc.pl
/nix/store/sah7h21jb4yvgm3vd1zrpr5k794qhmsz-perl-JSON-2.90.drv
/nix/store/ssr4zi7lb3ncv4k0w8b12fr46fxdkw4v-switch-to-configuration.pl
/nix/store/v0bnkqpay5ch19s43spgcdf1nrkh9akb-firmware.drv
/nix/store/yrsrk6lxk6520q7c10hz1kkkngh50msf-perl-File-Slurp-9999.19.drv
/nix/store/y3d7hy3nrhbbqf8c653ja914m3q7v5m1-install-grub.sh.drv

$ nix-store -r $(nix-store -q --references $(nix-instantiate -E "with import $PWD/nixos {}; system"))
warning: you did not specify ‘--add-root’; the result might be removed by the garbage collector
warning: you did not specify ‘--add-root’; the result might be removed by the garbage collector
/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh
/nix/store/21bglbr3a8mi0rqiqcm5vjszdjwsixkq-bash-4.4-p5-dev
/nix/store/g6farv9l1zvvazv2kwj9hxjpk2kyi8rs-bash-4.4-p5-doc
/nix/store/hj2vq5kckfammbzclj5p01qin8v63ibv-bash-4.4-p5
/nix/store/vk0jcc4dml9jmh9nr50000bc4s6iz36h-bash-4.4-p5-info
/nix/store/99j1brbgy8jahj5sq21qc3p721k2m067-bash-4.4-p5-info
/nix/store/9lgy5yd5wgn8w3rwgvkdnyhy63j0s1sg-bash-4.4-p5-dev
/nix/store/lr4p36k0yv6jl267cg2a3mvkxn3i56vf-bash-4.4-p5
/nix/store/r41d21xmvc077j5ada7lw9r4ph015bfm-bash-4.4-p5-doc
/nix/store/22j5siqnj7mqdnc95sjqrwx472lrc9rd-linux-pam-1.2.1-doc
/nix/store/5snq522y7ch4a9jbj0rvmyh93m1gbwkg-linux-pam-1.2.1-man
/nix/store/ca77amlsslfb1nkb2lsymbflhk474fl8-linux-pam-1.2.1
/nix/store/a5gvhlwrday3dj8z3v09nr65ngn5jzq3-glibc-2.25
/nix/store/dw8jn6m8zizr7p92kq6wh06jf7l265z6-glibc-2.25-static
/nix/store/f3r3vl31zdr59rppxxlzavl2iw0hh28z-glibc-2.25-bin
/nix/store/pqqmm39y5qar550nwab2ndszb6m02c82-glibc-2.25-dev
/nix/store/y9csq85mmpjydrqkm57lgzlvi6in0z42-glibc-2.25-debug
/nix/store/4g12477iwgjhxhpym3zb34qyp2x3lqjl-coreutils-8.26
/nix/store/zfmzyvnqc0xcqndmm4irzix6hy1bk1nk-coreutils-8.26-info
/nix/store/h8m78sqhyczqdb7kp035waa9wi78dvmc-perl-5.22.2-man
/nix/store/jswnnkp5lcy0zd2x2yqf0sipxf8fiz0j-perl-5.22.2
/nix/store/y326z16s7xd9v6s3yb06b6vl6pqgfvqj-perl-5.22.2-devdoc
/nix/store/a1z4byr3814ysir1q3ffhf0zxw0rf4gn-libcap-2.25
/nix/store/cbcwwg7zgd7jpalqd6jb9dagmwk13l84-libcap-2.25-pam
/nix/store/lb9cymj6k4nq0826g4nsa69bqxrdcsk6-libcap-2.25-lib
/nix/store/sj3dqg9mh4qwq6x9iahj0gxs2mjry9cv-libcap-2.25-dev
/nix/store/vrsl3is23ixrp6j921xidrdjc750xzy6-libcap-2.25-doc
/nix/store/x523malaivg4dazwi2vmgfkcf91i2n5j-kmod-23
/nix/store/a56rhy4awygqb9zdyc91ba5dsw55y2fr-systemd-232-man
/nix/store/v6gg8079bv7p8d4has9zg5ny9l91srg4-systemd-232-dev
/nix/store/ygcznr294l2w6flzvzagsicb25yyiwsy-systemd-232-lib
/nix/store/zs66inz96fpr0kkq0vaq17hdwd4s16ac-systemd-232
/nix/store/10irzz0j9r8wkp8x5l76v4h1p0yqhssi-polkit-0.113
/nix/store/kcfsa7421qnndsww8zxgx7ly4nlyhl7n-polkit-0.113-bin
/nix/store/nkc3ynbb751qlgqgaica50xi55m06bhd-polkit-0.113-dev
/nix/store/vihq9cq4yl97v81zfk7yhhr6nvsbhvhx-net-tools-1.60_p20120127084908
/nix/store/531giy6mqh207cadkffsi1z960l4jywz-update-users-groups.pl
/nix/store/dchg81xgpqfp41xpi82fy3nd3inkirpj-gnugrep-3.0
/nix/store/k0hh0hvc9xw1wkikcxzv6q5np3fkh15z-gnugrep-3.0-info
/nix/store/82lcndzfvz390y1aq0fwdjb6n7p6icp5-findutils-4.6.0-info
/nix/store/dpdnk0x016r5ybh2s4lysi5c8mjyyh2c-findutils-4.6.0
/nix/store/h4i1hj1j25nbcw461hpy8kk2140gqrni-stdenv
/nix/store/7x2ibrlp6696116syxsa0nli4gw18gkl-mounts.sh
/nix/store/10yb4x95fpbgvqr736f9fxxdlvn53bw5-dbus-1.10.16-lib
/nix/store/6jcy12d9ykfcbjqxdhmz04962p17kv44-dbus-1.10.16-dev
/nix/store/bq60mic98xzi74xq5gvj22a7wng9vnbd-dbus-1.10.16
/nix/store/pl49smb2f4vrxskwv4gs52p3cr83jpgr-dbus-1.10.16-doc
/nix/store/p49kjzq5xvyf7bcgqcfdczvmy0zhyrjn-etc
/nix/store/228wxsn7ycv1v2p0yc3s5wdjviyci28q-shadow-4.4
/nix/store/irpkn9k2n25vb6gn32kspxmdr94wji1f-shadow-4.4-man
/nix/store/m0hxkjg1pv9xajy0aqjlhmfr04dvwcvx-shadow-4.4-su
/nix/store/h5frbby9h9pgzrzjwjvsj6i1iqqkwl7p-util-linux-2.29-dev
/nix/store/icpwbg8pymafdw1vxni4in12mg5sr4kb-util-linux-2.29-man
/nix/store/pmzc56afmvvxwxavp50mnav0ccm52px4-util-linux-2.29-bin
/nix/store/vy6csiz57nnqlnayslpbvi0nwpb9kgy8-util-linux-2.29
/nix/store/aga2652jafdl23xsgzz9qv13v758i06p-openresolv-3.8.1
/nix/store/chn62622lqlykwb2nrmvqii31m3w5z9r-stage-2-init.sh
/nix/store/2mxmpyh40q860csd5mkix98w4d1f7v48-iputils-20161105
/nix/store/fnrk6p60mnjgd3al4iainprskq97z6jr-initrd
/nix/store/54460ghjzr7lyarnaizawgwcvbnsdhq4-linux-4.9.12
/nix/store/d7vkxm1psdwv1h8k4yp49ailyqvf5ny2-linux-4.9.12-dev
/nix/store/7hvrrn2p0d26amasx78bbmzi5pbgr1k8-users-groups.json
/nix/store/g96rqfidjdwws0nysvl42ywxsvzjdxxk-e2fsprogs-1.43.4-man
/nix/store/l5kry4amjhby0ladamd1ccy9m03xfp3z-e2fsprogs-1.43.4
/nix/store/x2zi2f45vvrsqcw63vkya9ighp5qlij0-e2fsprogs-1.43.4-dev
/nix/store/xgyrz3xz8q7sk8lhlihwvjy8ax00hb80-e2fsprogs-1.43.4-bin
/nix/store/sgw7jnji7fclpfw7kn049ilrnsrn06x5-sudo-1.8.19p2
/nix/store/ilavi17kapf64smxkqi2cbpyxj2vs0n1-kernel-modules
/nix/store/6ikf05g7bgg3qi5srq4pjl7aimczd6fz-security-wrapper
/nix/store/009wzipg9nk1lw8hv197l4z9rzb4g5ad-fuse-2.9.7
/nix/store/fgl18yqa7y1qw3zrnc818bfzs34hpwk5-system-path
/nix/store/r620i2hsldg2ssr3nijdimyw1wyl2wlp-setup-etc.pl
/nix/store/adliys850sckaql547cklas6wv8hhfpk-perl-JSON-2.90
/nix/store/cf8106bjqyqlz4f7gwk7qgm3wdbz4j8c-perl-JSON-2.90-devdoc
/nix/store/ssr4zi7lb3ncv4k0w8b12fr46fxdkw4v-switch-to-configuration.pl
/nix/store/zxmmdqzg6saaainrpjxcj146snr6dkx8-firmware
/nix/store/nwffffb65f637zdw635kyb7jbi5qf2s7-perl-File-Slurp-9999.19
/nix/store/q95fz1pz9wpbldcsfwbllk8yg49dwqx1-perl-File-Slurp-9999.19-devdoc
/nix/store/nyd0mbr8qsmr1hdd128ff7c49sxx8ccf-install-grub.sh

$ nix-store --export $(nix-store -r $(nix-store -q --references $(nix-instantiate -E "with import $PWD/nixos {}; system"))) > build.closure

Will build.closure not contain what I want?

shlevy commented 7 years ago

How are you going to do all these nix-store -q's from a build?

copumpkin commented 7 years ago

In the short term, I'm just trying to get https://github.com/NixOS/nixpkgs/pull/23026 to work like it used to. The separation I envision allows nixos-install to speak to Nix store/db on the host, so it can run all those commands, compute the proper closure, and then dump it into the target machine.

I'd still leave this open because I want to be able to do all this from a pure image/tarball build too, but I want to get my nixos-install refactor merged first, and to do that, I need the installer tests to pass 😄

shlevy commented 7 years ago

Oh, well I didn't look at the specific query sequence you performed here, but of course you can just query references and get their derivers.

copumpkin commented 7 years ago

Actually, I'm not sure my command is correct. I think it'll give me the immediate build-time references but not their runtime closures. If you know of a simple way to do the runtime closures of them without bash loops, I'm all ears 😄

Edit: I can just shove a nix-store -qR into the chain, of course. Edit 2: The resulting closure seems huge 😱 About 1.1gb for my simple test.

7c6f434c commented 7 years ago

«Without bash loops»… Hm, he wants an xargs showoff!

I think this:

drv="$( nix-instantiate '<nixpkgs>' -A hello )"
drvRefs="$( echo "$drv" | xargs nix-store -q --references )" 
( echo "$drvRefs" | grep '[.]drv$' | xargs nix-store -q --outputs ; 
  echo "$drvRefs" | grep -v '[.]drv$' ) | 
xargs nix-store -r | xargs nix-store -qR

works correctly.

7c6f434c commented 7 years ago

It gives 687MiB for systemd alone.

I think part of the problem with the closure size is that -q --references doesn't say which outputs of the dependencies are referred, i.e. man output is not often used by builders, but we get everything in the closure.

shlevy commented 7 years ago

You can import a .drv in nix and it resolves to a derivation that you can then use as a buildInput or whatever.

copumpkin commented 7 years ago

@7c6f434c hmm, what's wrong with this?

    expr="(import <nixpkgs/nixos> {}).system"
    base_closure="$(nix-store -qR $(nix-build -E "$expr"))"
    build_closure="$(nix-store -qR $(nix-store -r $(nix-store -q --references $(nix-instantiate -E "$expr"))))"
    nix-store --export $base_closure $build_closure > $system_closure

And yeah, the specific output issue is a problem if I do it this way. I wish we had more machinery for manipulating aterm/.drv files. If I peek at the .drv file I can see the specific outputs needed...

@shlevy can you give an example? Not sure I follow

7c6f434c commented 7 years ago

Hm, I added nix-store -r too late to notice it simplifies the other part. Nothing wrong, I just write simple shell stuff interactively and with the minimal amount of variables possible.

shlevy commented 7 years ago
nix-instantiate -A hello '<nixpkgs>' | xargs nix-instantiate -E '{ hello }: let pkgs = import <nixpkgs> {}; in pkgs.runCommand "say-hello" { buildInputs = [ (import hello) ]; } "hello > $out"' --arg hello
copumpkin commented 7 years ago

@7c6f434c aha, thanks!

@shlevy oh, interesting! Hmm, so in that case, is there any downside to the attrset exposed by importing a .drv file exposing the inputs section of the Derive aterm thingy? That seems like a dead-simple way to introspect inputs of a derivation.

shlevy commented 7 years ago

Only downside is someone has to write and merge it :stuck_out_tongue: Also less useful than a general context introspection functionality, but :man_shrugging:

copumpkin commented 7 years ago

Yeah, I'm mostly suggesting this because it seems like a simple extension of existing behavior (and therefore uncontroversial), unless I'm missing something. I think I could make one smallish C++ change and one small Nix change and be able to do exactly what I wanted in pure Nix.

copumpkin commented 7 years ago

Maybe I oversimplified. If I import a .drv file in my local Nix build, I now get inputDrvs and inputSrcs attributes with what you'd expect inside them, but I (obviously in retrospect) can't reliably do something like inst/bin/nix-instantiate --eval -E "import ((import <nixpkgs> {}).hello.drvPath)" unless I instantiate hello first. Easy to do on the command line, but I don't know how to get similar behavior in pure Nix. I could probably use seq, but I don't know how to force a .drv to be produced (rather than being substituted)

shlevy commented 7 years ago

Pass --read-write-mode

shlevy commented 7 years ago

(only needed if doing nix-instantiate --eval, will work fine already in a normal nix eval that's not readonly)

shlevy commented 7 years ago

Also, you probably want builtins.unsafeDiscardOutputDependency so the import doesn't require you to realize the build.

copumpkin commented 7 years ago

So I almost have the installer test for https://github.com/NixOS/nixpkgs/pull/23026 working again (given that we need the build-time closure), but it wants to build everything from scratch inside the VM 😦

aszlig commented 6 years ago

@copumpkin: A somewhat very hacky way (you asked for it: "Not even possible with horrifying hackery?") to do this could be something like this (reading the contents of the .drv and searching for all store paths contained in the .drv contents):

{ pkg, lib ? import <nixpkgs/lib> }: let
  drv = builtins.readFile pkg.drvPath;
  storeDirRe = lib.replaceStrings [ "." ] [ "\\." ] builtins.storeDir;
  storeBaseRe = "[0-9a-df-np-sv-z]{32}-[+_?=a-zA-Z0-9-][+_?=.a-zA-Z0-9-]*";
  re = "(${storeDirRe}/${storeBaseRe}\\.drv)";
  inputs = lib.concatLists (lib.filter lib.isList (builtins.split re drv));
in map import inputs

Test output with the contents above in gather.nix:

$ nix-instantiate --eval --strict -E 'map toString (import ./gather.nix { pkg = (import <nixpkgs> {}).hello; })'
[ "/nix/store/8w4m980qvndxbvx32s0h5w0x6a9828yi-bash-4.4-p23-dev" "/nix/store/bcrwa5b8z54xrl95ipfrq6nqhb0530yi-stdenv-linux" "/nix/store/3x7dwzq014bblazs7kq20p9hyzz0qh8g-hello-2.10.tar.gz" ]
$
nmattia commented 4 years ago

Came across this by chance, this might be useful: https://www.nmattia.com/posts/2019-10-08-runtime-dependencies.html

nixos-discourse commented 4 years ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/any-way-to-get-a-derivations-inputdrvs-from-within-nix/7212/3

domenkozar commented 4 years ago

This would be useful to cover in the new CLI UI.

lorenzleutgeb commented 3 years ago

I ran into this as well, with additional complications. The setup is as follows:

The first issue is getting the store derivation from the Flake, which IMO is more closely related or even covered by #3908 and off-topic here. The second issue is to compute what I would call the "realization closure", meaning exactly what's being discussed here (the runtime closure of the build-time dependencies of the package), in order to do a nix-store --export and cache this across builds.

Note that the term "build-time dependencies" is used in nix-store --help but I think it's not what is being asked for here:

       Print the build-time dependencies of svn:

       $ nix-store -qR $(nix-store -qd $(which svn))
       /nix/store/02iizgn86m42q905rddvg4ja975bk2i4-grep-2.5.1.tar.bz2.drv
       /nix/store/07a2bzxmzwz5hp58nf03pahrv2ygwgs3-gcc-wrapper.sh
       /nix/store/0ma7c9wsbaxahwwl04gbw3fcd806ski4-glibc-2.3.4.drv
       ... lots of other paths ...

       The difference with the previous example is that we ask the closure of the derivation (-qd), not
       the closure of the output path that contains svn.

       Show the build-time dependencies as a tree:

       $ nix-store -q --tree $(nix-store -qd $(which svn))
       /nix/store/7i5082kfb6yjbqdbiwdhhza0am2xvh6c-subversion-1.1.4.drv
       +---/nix/store/d8afh10z72n8l1cr5w42366abiblgn54-builder.sh
       +---/nix/store/fmzxmpjx2lh849ph0l36snfj9zdibw67-bash-3.0.drv
       |   +---/nix/store/570hmhmx3v57605cqg9yfvvyh0nnb8k8-bash
       |   +---/nix/store/p3srsbd8dx44v2pg6nbnszab5mcwx03v-builder.sh
       ...

I ended up doing

nix-store --query --references $(nix eval --raw .#defaultPackage.x86_64-linux.drvPath) \
 | xargs nix-store --realise \
 | xargs nix-store --query --requisites

inspired by NixOS/nixpkgs#95191

stale[bot] commented 3 years ago

I marked this as stale due to inactivity. → More info

nixos-discourse commented 2 years ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-export-binary-packages-of-all-build-inputs/17023/2

stale[bot] commented 2 years ago

I marked this as stale due to inactivity. → More info

nixos-discourse commented 5 months ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/potential-rfc-idea-expansion-of-the-dependency-tracking-system/43418/1