Issues buliding crates that invoke pkg-config in build.rs

[mitchmindtree]

Hey, thanks for all your work on such a cool tool!

I'm frequently creating new projects that have quite large dependencies, and often the initial build can take a good 15+ minutes. I'm excited to see how well crate2nix can help with taking advantage of the nix store as a cache for crate build artifacts.

I've tried running it on a few projects with varying success. Some simpler projects work nicely, but I'm having issues with more sophisticated builds (e.g. for GUI applications). Most often these errors seem to have some relation to pkg-config not being able find some library when invoked in a build.rs script, or something along these lines.

Here's an example attempting at building nannou:

---

Generated Cargo.nix

Cargo.nix.txt

(added .txt so that github would allow upload)

Command

nix-build -E "with import <nixpkgs> {}; let c = callPackage ./Cargo.nix {}; in c.workspaceMembers.\"nannou\".build"

Output

``` these derivations will be built: /nix/store/p20mvfg5nwhpgr0yjjw456infkkmplq5-xml-rs-0.8.3.tar.gz.drv /nix/store/2ajm7ky62zjaxnjzjj9d09b8b4568638-rust_xml-rs-0.8.3.drv /nix/store/qhx8hm7a87bll22b8zk3zs3c15qxgdki-rust_wayland-sys-0.28.5.drv /nix/store/nfnw4gk46jsch0kqcr6qcy67v3ffk2ky-rust_wayland-commons-0.28.5.drv /nix/store/qplvn2i1wyy6vv30h0ssj13pfzxkjnk0-rust_quote-1.0.9.drv /nix/store/v7xbypjc7bsk55qj7z1c0zp0ycrpmdmp-rust_wayland-scanner-0.28.5.drv /nix/store/6y09w1nndbcnh42gk9ks5ny0c9h4x401-rust_wayland-client-0.28.5.drv /nix/store/bdq1jr85r3908gwzrw1wbyb50hpacdy2-xcursor-0.3.3.tar.gz.drv /nix/store/dhfg7p88awi9bm5bdf05cr3h27x90dn4-rust_xcursor-0.3.3.drv /nix/store/6ffkyfimq3xmjmr5x6zlvynfn41fi38c-rust_wayland-cursor-0.28.5.drv /nix/store/bd7yy2r53531pf1mif6yk7d12lm43g96-xdg-2.2.0.tar.gz.drv /nix/store/llmvjn5111jfwfyd89xfslawihskzwii-rust_xdg-2.2.0.drv /nix/store/y0q2ac2l4s5c3v1ckjrjgmx58b849fi8-rust_rusttype-0.9.2.drv /nix/store/pm2agzlpx1zbn8pl8np83l1091x8y8yl-rust_andrew-0.3.1.drv /nix/store/rz8yf9p4bnaqf2nrjall3rbjc200r7qy-rust_wayland-protocols-0.28.5.drv /nix/store/0dzjfmq3nkdvl7gqsgkbbdzm2ibpdkpy-rust_smithay-client-toolkit-0.12.2.drv /nix/store/10p7mcy6lpji7h6sazf0knmv9jich2wb-xcb-0.9.0.tar.gz.drv /nix/store/ixg776yamavmbamh04q4z7hiagczhzrp-rust_syn-1.0.60.drv /nix/store/23z7dxh0zgki80n56k2vjbaqc69f9wsp-rust_serde_derive-1.0.123.drv /nix/store/7jk2bp49lqyqr0pjhbh1i4g695lm8m89-rust_serde-1.0.123.drv /nix/store/4g9jcsj0jc4qy8jk8chplas9h5jlikzy-rust_quote-0.6.13.drv /nix/store/gxbdr8c8a80rkrzbjzvqwlpvs29696w9-rust_wayland-scanner-0.23.6.drv /nix/store/xd8hb99c89y009lfhmcb2pydsnmbynf2-rust_wayland-commons-0.23.6.drv /nix/store/msklw46xv8mnzqgdbmwfadsllikykrs1-rust_wayland-client-0.23.6.drv /nix/store/ijj033mk6giyy8kaf628aqak59mx9knh-rust_wayland-protocols-0.23.6.drv /nix/store/xjppm2dia73r6wj44lbd8spk87xa7n6v-rust_andrew-0.2.1.drv /nix/store/cpx0sgcplp309krjbbv3dhkck50wd0pk-rust_smithay-client-toolkit-0.6.6.drv /nix/store/gb1pffjmlajb3afwffylz4amv2glk5lg-rust_smithay-clipboard-0.4.0.drv /nix/store/rmm4nyrfvbvj4cfgb72774dvi0by9294-rust_xcb-0.9.0.drv /nix/store/p8n3ma8lj1r9zmlcglgcagjlx2nfsci8-rust_x11-clipboard-0.5.1.drv /nix/store/qcdkilxzfpfnnzmrg4bhdvchxgrm0mq8-rust_conrod_derive-0.71.0.drv /nix/store/rppabp725w62fjbmzq5avhgyiygbjw8f-rust_copypasta-0.6.3.drv /nix/store/s6gi1dc1lfawp9rvicxhml2jjdiszj9p-rust_pistoncore-input-1.0.0.drv /nix/store/475wd7mw07mzr7zbb3gsjqx8dj1zkfca-rust_conrod_core-0.71.0.drv /nix/store/495px3lxr4q90ffzdl4fqk19157pm3i7-rust_gif-0.11.1.drv /nix/store/6dbd49rln2xia76ah3wbpbccg4qimzy1-rust_futures-macro-0.3.13.drv /nix/store/9wa2kzgycqi6yimy47an11k274yij9ip-rust_futures-util-0.3.13.drv /nix/store/r9lil0fljglh6ywdrr14rd2ns798yjww-rust_x11-2.18.2.drv /nix/store/czvm5nl7xl7h43hw6cb4mbpvcc9jxr7h-rust_gfx-backend-vulkan-0.6.5.drv /nix/store/k07k3gmiqb01z8784jpq05xc5rh2iriv-rust_thiserror-impl-1.0.24.drv /nix/store/jdgl7f1p83k7l130nr76izlr4p8gnfr0-rust_thiserror-1.0.24.drv /nix/store/kdza6qaibcis73465yxwzfk6zjm3b2ci-rust_futures-executor-0.3.13.drv /nix/store/p6mfgqjzqicgwdhzsylv3sgk1pdrls90-rust_futures-0.3.13.drv /nix/store/wdq83837m9p21i8zlplxd8cczli173r9-rust_naga-0.2.0.drv /nix/store/pdxj7n7q8diaq7ys6fdyvw58gsfj6dnd-rust_wgpu-core-0.6.5.drv /nix/store/pjl2ifxgyfdm6d69hh1016w29dd9zdy9-rust_wgpu-0.6.2.drv /nix/store/547xzbscgidhn6ncdz2f1j1g2dpkhndz-rust_conrod_wgpu-0.71.0.drv /nix/store/ddf7qmb4f8cxh61wkydh47604mqqgdr5-rust_image-0.18.0.drv /nix/store/bfxxxzppsfnd7rm2mbljmnqsyz7i0z1y-rust_noise-0.6.0.drv /nix/store/c1pmffqii024c08bxyhcnz07920ygif5-rust_serde_json-1.0.63.drv /nix/store/i5sr5fbbxmx1mpszc053z1n1acvs4ix6-rust_palette_derive-0.5.0.drv /nix/store/w705s2w3m5ih544cilnz577mfpr2iii8-rust_tiff-0.6.1.drv /nix/store/if9ksac87xypb4gifr8asdc68jh50j5x-rust_image-0.23.13.drv /nix/store/5x7v3rjlr6xwazb89k6pjvhfwa3yqghh-x11-dl-2.18.5.tar.gz.drv /nix/store/j6w384lqj7z8rmn56gdlnpjslbbmv4x6-rust_x11-dl-2.18.5.drv /nix/store/kz3sbybxys22w66wsspavlbn2x1k3m23-rust_winit-0.24.0.drv /nix/store/l8sxhwiyzjpz3xqsqcf8c9y7ky5w1zv3-rust_palette-0.5.0.drv /nix/store/lrwy524x09lzmzppamdm97qpd1jhrwbw-rust_toml-0.5.8.drv /nix/store/v45jcwyz1wfp66qzb62fkh3wgam0w5pg-rust_cgmath-0.17.0.drv /nix/store/1xb54ih0adj5192hlga5vm7i01163ps3-rust_nannou-0.15.0.drv building '/nix/store/495px3lxr4q90ffzdl4fqk19157pm3i7-rust_gif-0.11.1.drv'... building '/nix/store/ddf7qmb4f8cxh61wkydh47604mqqgdr5-rust_image-0.18.0.drv'... building '/nix/store/qplvn2i1wyy6vv30h0ssj13pfzxkjnk0-rust_quote-1.0.9.drv'... building '/nix/store/y0q2ac2l4s5c3v1ckjrjgmx58b849fi8-rust_rusttype-0.9.2.drv'... building '/nix/store/w705s2w3m5ih544cilnz577mfpr2iii8-rust_tiff-0.6.1.drv'... building '/nix/store/xd8hb99c89y009lfhmcb2pydsnmbynf2-rust_wayland-commons-0.23.6.drv'... building '/nix/store/qhx8hm7a87bll22b8zk3zs3c15qxgdki-rust_wayland-sys-0.28.5.drv'... building '/nix/store/r9lil0fljglh6ywdrr14rd2ns798yjww-rust_x11-2.18.2.drv'... unpacking sources unpacking source archive /nix/store/w4p2y5lnhv18vck1jpkn8zbrlc4svn4m-gif-0.11.1.tar.gz source root is gif-0.11.1 setting SOURCE_DATE_EPOCH to timestamp 1601060447 of file gif-0.11.1/Cargo.toml patching sources configuring Running cd . unpacking sources unpacking source archive /nix/store/wd0m5jpcz6jbjzqrkxb2whnvcs9dj6ch-image-0.18.0.tar.gz source root is image-0.18.0 unpacking sources unpacking source archive /nix/store/q0cjwg7lqq89309vw1bbjp5v4zjc3mnk-quote-1.0.9.tar.gz setting SOURCE_DATE_EPOCH to timestamp 1512275229 of file image-0.18.0/src/utils/lzw.rs source root is quote-1.0.9 patching sources configuring Running cd . patching sources unpacking sources configuring unpacking source archive /nix/store/brb92apn9v2m95p2c8yrnrq6p7j5dnln-rusttype-0.9.2.tar.gz Running cd . source root is rusttype-0.9.2 building unpacking sources setting SOURCE_DATE_EPOCH to timestamp 1590192284 of file rusttype-0.9.2/Cargo.toml unpacking source archive /nix/store/an0irsbjrnc46nfncgwmrkr0vpvalvcc-tiff-0.6.1.tar.gz patching sources configuring Running cd . Building src/lib.rs (gif) Running rustc --crate-name gif src/lib.rs --out-dir target/lib -L dependency=target/deps --cap-lints allow -C opt-level=3 -C codegen-units=8 --remap-path-prefix=/build=/ --extern color_quant=/nix/store/jd9b99d8c9h6xmzihwy1sd6j6arn2hyg-rust_color_quant-1.1.0-lib/lib/libcolor_quant-6606febe43.rlib --extern weezl=/nix/store/4pg315c0ilwkw50nlb70s9i3i7997xqc-rust_weezl-0.1.4-lib/lib/libweezl-976a221ae2.rlib --cfg feature="default" --cfg feature="raii_no_panic" --cfg feature="std" --edition 2018 -C metadata=c44e48269e -C extra-filename=-c44e48269e --crate-type lib --color always unpacking sources unpacking source archive /nix/store/zck7n5phlff6gh8ycicpc8dhp2mp9jhx-wayland-commons-0.23.6.tar.gz source root is wayland-commons-0.23.6 building unpacking sources unpacking source archive /nix/store/9xjhng5flvlmvlfpyhh0whzh4mg0d1ap-wayland-sys-0.28.5.tar.gz setting SOURCE_DATE_EPOCH to timestamp 1567783328 of file wayland-commons-0.23.6/Cargo.toml.orig source root is tiff-0.6.1 patching sources source root is wayland-sys-0.28.5 configuring Building src/lib.rs (quote) Running cd . setting SOURCE_DATE_EPOCH to timestamp 1607800999 of file tiff-0.6.1/Cargo.toml Running rustc --crate-name quote src/lib.rs --out-dir target/lib -L dependency=target/deps --cap-lints allow -C opt-level=3 -C codegen-units=8 --remap-path-prefix=/build=/ --extern proc_macro2=/nix/store/57yfxx9ysm6x4s9p1ddxaqa7mqs7r8sc-rust_proc-macro2-1.0.24-lib/lib/libproc_macro2-1d9c1361c7.rlib --cfg feature="default" --cfg feature="proc-macro" --edition 2018 -C metadata=f98928fce8 -C extra-filename=-f98928fce8 --crate-type lib --color always unpacking sources unpacking source archive /nix/store/xcrmq1shvy93s9waidmks6dfc8g8z9wn-x11-2.18.2.tar.gz patching sources configuring patching sources configuring Running cd . source root is x11-2.18.2 Running cd . building setting SOURCE_DATE_EPOCH to timestamp 1580027074 of file x11-2.18.2/Cargo.toml patching sources configuring Building src/lib.rs (rusttype) Running cd . Running rustc --crate-name rusttype src/lib.rs --out-dir target/lib -L dependency=target/deps --cap-lints allow -C opt-level=3 -C codegen-units=8 --remap-path-prefix=/build=/ --extern ab_glyph_rasterizer=/nix/store/pr3djyfv9jadqv97r67agvxbdf1a19cz-rust_ab_glyph_rasterizer-0.1.4-lib/lib/libab_glyph_rasterizer-bc19335483.rlib --extern owned_ttf_parser=/nix/store/vmicp30qfc80n22yil4nkkybsr7pa5f4-rust_owned_ttf_parser-0.6.0-lib/lib/libowned_ttf_parser-f0ac2aeb4c.rlib --cfg feature="default" --cfg feature="has-atomics" --cfg feature="std" --edition 2018 -C metadata=47b20ed39c -C extra-filename=-47b20ed39c --crate-type lib --color always Building build.rs (x11) Building build.rs (wayland-sys) Running rustc --crate-name build_script_build build.rs --crate-type bin -C opt-level=3 -C codegen-units=8 --edition 2015 --cfg feature="default" --cfg feature="xlib" --out-dir target/build/x11 --emit=dep-info,link -L dependency=target/buildDeps --extern pkg_config=/nix/store/pj9bjyzvpx6sixss5l11q9ahldc84iql-rust_pkg-config-0.3.19-lib/lib/libpkg_config-b69474cff7.rlib --cap-lints allow --color always building Running rustc --crate-name build_script_build build.rs --crate-type bin -C opt-level=3 -C codegen-units=8 --edition 2018 --cfg feature="client" --cfg feature="default" --cfg feature="dlib" --cfg feature="dlopen" --cfg feature="lazy_static" --out-dir target/build/wayland-sys --emit=dep-info,link -L dependency=target/buildDeps --extern pkg_config=/nix/store/pj9bjyzvpx6sixss5l11q9ahldc84iql-rust_pkg-config-0.3.19-lib/lib/libpkg_config-b69474cff7.rlib --cap-lints allow --color always Building src/lib.rs (wayland-commons) Running rustc --crate-name wayland_commons src/lib.rs --out-dir target/lib -L dependency=target/deps --cap-lints allow -C opt-level=3 -C codegen-units=8 --remap-path-prefix=/build=/ --extern nix=/nix/store/090na2wvbnwh9mzk9p3viy7in7yy27nv-rust_nix-0.14.1-lib/lib/libnix-094972b93c.rlib --extern wayland_sys=/nix/store/ify1dsqa6df57kr9289a4qsxc6zlm6am-rust_wayland-sys-0.23.6-lib/lib/libwayland_sys-ce559b0c35.rlib --cfg feature="default" --edition 2015 -C metadata=b818b0de97 -C extra-filename=-b818b0de97 --crate-type lib --color always building Building src/lib.rs (tiff) Running rustc --crate-name tiff src/lib.rs --out-dir target/lib -L dependency=target/deps --cap-lints allow -C opt-level=3 -C codegen-units=8 --remap-path-prefix=/build=/ --extern jpeg=/nix/store/bj3pqcmfr2vc6h2wilnxcara88jwxa2d-rust_jpeg-decoder-0.1.22-lib/lib/libjpeg_decoder-66aee251a1.rlib --extern miniz_oxide=/nix/store/8yply9b0hzgszja28ykm4m5lbjfpiy18-rust_miniz_oxide-0.4.4-lib/lib/libminiz_oxide-1f1ac096ff.rlib --extern weezl=/nix/store/4pg315c0ilwkw50nlb70s9i3i7997xqc-rust_weezl-0.1.4-lib/lib/libweezl-976a221ae2.rlib --cfg feature="default" --edition 2018 -C metadata=70ff8fbb09 -C extra-filename=-70ff8fbb09 --crate-type lib --color always building Building src/lib.rs (image) Running rustc --crate-name image src/lib.rs --out-dir target/lib -L dependency=target/deps --cap-lints allow -C opt-level=3 -C codegen-units=8 --remap-path-prefix=/build=/ --extern byteorder=/nix/store/0lf9hk95s3k166pc9yb5z5gpihz37gni-rust_byteorder-1.4.2-lib/lib/libbyteorder-a77de4627c.rlib --extern enum_primitive=/nix/store/y85va04cczrsqcanc2r860q2h2d5p6lr-rust_enum_primitive-0.1.1-lib/lib/libenum_primitive-ff25acb1df.rlib --extern gif=/nix/store/jacwj2mxqsp4mrbs7mazc5nxpbxkxzvh-rust_gif-0.9.2-lib/lib/libgif-80e536f365.rlib --extern jpeg_decoder=/nix/store/bj3pqcmfr2vc6h2wilnxcara88jwxa2d-rust_jpeg-decoder-0.1.22-lib/lib/libjpeg_decoder-66aee251a1.rlib --extern num_iter=/nix/store/l76cgyw1fvk7i07sl00k5qfanllw2a8w-rust_num-iter-0.1.42-lib/lib/libnum_iter-2f42add999.rlib --extern num_rational=/nix/store/qxkmyhjykli0h3q0pgx10fgbda1f9pyg-rust_num-rational-0.1.42-lib/lib/libnum_rational-166f03fdad.rlib --extern num_traits=/nix/store/4d4k4dizk1cf6lcnllw6xic11isprmgq-rust_num-traits-0.1.43-lib/lib/libnum_traits-e6de60084a.rlib --extern png=/nix/store/bghk7sfnbzsqmh0ichyrp7031mf346ar-rust_png-0.11.0-lib/lib/libpng-8aa4e7f888.rlib --extern scoped_threadpool=/nix/store/apg4lfa80jajwyda6i68zm7p87bsgd86-rust_scoped_threadpool-0.1.9-lib/lib/libscoped_threadpool-c66b0f7747.rlib --cfg feature="bmp" --cfg feature="default" --cfg feature="gif" --cfg feature="gif_codec" --cfg feature="hdr" --cfg feature="ico" --cfg feature="jpeg" --cfg feature="jpeg-decoder" --cfg feature="png" --cfg feature="png_codec" --cfg feature="pnm" --cfg feature="ppm" --cfg feature="scoped_threadpool" --cfg feature="tga" --cfg feature="tiff" --cfg feature="webp" --edition 2015 -C metadata=878ad819eb -C extra-filename=-878ad819eb --crate-type lib --color always building Building src/lib.rs (wayland-sys) Running rustc --crate-name wayland_sys src/lib.rs --out-dir target/lib -L dependency=target/deps --cap-lints allow -C opt-level=3 -C codegen-units=8 --remap-path-prefix=/build=/ --extern dlib=/nix/store/k7qv586q5jvhj7a0rmikhyayh5k6ajyv-rust_dlib-0.5.0-lib/lib/libdlib-4ec3c92ea8.rlib --extern lazy_static=/nix/store/8x6i3m1pkzwssby848lh64qp7m1h72sn-rust_lazy_static-1.4.0-lib/lib/liblazy_static-7c83871282.rlib --cfg feature="client" --cfg feature="default" --cfg feature="dlib" --cfg feature="dlopen" --cfg feature="lazy_static" --edition 2018 -C metadata=4f918235f7 -C extra-filename=-4f918235f7 --crate-type lib -L /build/wayland-sys-0.28.5/target/build/wayland-sys.out --color always thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Command { command: "\"pkg-config\" \"--libs\" \"--cflags\" \"x11\" \"x11 >= 1.4.99.1\"", cause: Os { code: 2, kind: NotFound, message: "No such file or directory" } }', build.rs:36:67 stack backtrace: 0: rust_begin_unwind 1: core::panicking::panic_fmt 2: core::result::unwrap_failed 3: build_script_build::main note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. builder for '/nix/store/r9lil0fljglh6ywdrr14rd2ns798yjww-rust_x11-2.18.2.drv' failed with exit code 101 cannot build derivation '/nix/store/1xb54ih0adj5192hlga5vm7i01163ps3-rust_nannou-0.15.0.drv': 1 dependencies couldn't be built error: build of '/nix/store/1xb54ih0adj5192hlga5vm7i01163ps3-rust_nannou-0.15.0.drv' failed ```

---

I can confirm that I can build the project successfully using the regular cargo build command. I'm calling the command inside a rust-dev-env.nix shell environment that should provide all the necessary dependencies as buildInputs (e.g. x11, pkg-config, etc). Perhaps the issue is that these buildInputs are not forwarded to the nix-build command above? Do I have to add those manually to the generated Cargo.nix somehow?

Any advice appreciated :)

[lblasc]

I have lot of deps with pkg-config, and they build fine, but you need to do something like this:

{ pkgs ? import <nixpkgs> { } }:

with pkgs;
let
c = import ./Cargo.nix {
    inherit pkgs;
    defaultCrateOverrides = pkgs.defaultCrateOverrides // {
      rust_x11 = attrs: {
        buildInputs = [ pkg-config someOtherMissingDep ];
      };
    };
  };
in c.workspaceMembers."nannou".build

[mitchmindtree]

Thanks, I clearly didn't read the README thoroughly enough! For future readers, there's a section on this here.