search

NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
17.69k stars 13.83k forks source link

Build failure: rPackages.terra likely due to pkg-config on Darwin #346425

Open b-rodrigues opened 5 days ago

b-rodrigues commented 5 days ago

Steps To Reproduce

Try to build rPackages.terra on Darwin (also affects other packages, such as rPackages.sf.

Build log

building '/nix/store/dza9i9zwxmnzzbwb0xchdksa9ari8dkw-r-terra-1.7-78.drv'...
Running phase: unpackPhase
unpacking source archive /nix/store/bl6pnawvyhfd7yv45s2vlchsj556n58h-terra_1.7-78.tar.gz
source root is terra
setting SOURCE_DATE_EPOCH to timestamp 1716388203 of file terra/MD5
Running phase: patchPhase
Running phase: configurePhase
Running phase: buildPhase
Running phase: checkPhase
Running phase: installPhase
During startup - Warning messages:
1: Setting LC_CTYPE failed, using "C" 
2: Setting LC_TIME failed, using "C" 
3: Setting LC_MESSAGES failed, using "C" 
4: Setting LC_MONETARY failed, using "C" 
* installing *source* package 'terra' ...
** package 'terra' successfully unpacked and MD5 sums checked
** using staged installation
configure: CC: clang
configure: CXX: /nix/store/bkhpl7q6fzghx08zzrmcckh12rrnnl29-clang-wrapper-16.0.6/bin/c++ -std=gnu++17
checking for gdal-config... /nix/store/zj7j809wk0c5iv5s7fsd32fnhh6185vr-gdal-3.9.2/bin/gdal-config
checking gdal-config usability... yes
configure: GDAL: 3.9.2
checking GDAL version >= 2.0.1... yes
checking for gcc... clang
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C... yes
checking whether clang accepts -g... yes
checking for clang option to enable C11 features... none needed
checking for stdio.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for strings.h... yes
checking for sys/stat.h... yes
checking for sys/types.h... yes
checking for unistd.h... yes
checking for gdal.h... yes
checking GDAL: linking with --libs only... yes
checking GDAL: /nix/store/zj7j809wk0c5iv5s7fsd32fnhh6185vr-gdal-3.9.2/share/gdal/pcs.csv readable... no
checking GDAL: checking whether PROJ is available for linking:... yes
checking GDAL: checking whether PROJ is available for running:... yes
configure: GDAL: 3.9.2
configure: pkg-config proj exists, will use it
configure: using proj.h.
Package libtiff-4 was not found in the pkg-config search path.
Perhaps you should add the directory containing `libtiff-4.pc'
to the PKG_CONFIG_PATH environment variable
Package 'libtiff-4', required by 'proj', not found
configure: PROJ: 9.4.1
checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no
configure: error: libproj or sqlite3 not found in standard or given locations.
ERROR: configuration failed for package 'terra'
* removing '/nix/store/nbwfdhv18l9c3k2zby6ni880x51py2gl-r-terra-1.7-78/library/terra'
error: builder for '/nix/store/dza9i9zwxmnzzbwb0xchdksa9ari8dkw-r-terra-1.7-78.drv' failed with exit code 1;
       last 25 log lines:
       > checking for stdlib.h... yes
       > checking for string.h... yes
       > checking for inttypes.h... yes
       > checking for stdint.h... yes
       > checking for strings.h... yes
       > checking for sys/stat.h... yes
       > checking for sys/types.h... yes
       > checking for unistd.h... yes
       > checking for gdal.h... yes
       > checking GDAL: linking with --libs only... yes
       > checking GDAL: /nix/store/zj7j809wk0c5iv5s7fsd32fnhh6185vr-gdal-3.9.2/share/gdal/pcs.csv readable... no
       > checking GDAL: checking whether PROJ is available for linking:... yes
       > checking GDAL: checking whether PROJ is available for running:... yes
       > configure: GDAL: 3.9.2
       > configure: pkg-config proj exists, will use it
       > configure: using proj.h.
       > Package libtiff-4 was not found in the pkg-config search path.
       > Perhaps you should add the directory containing `libtiff-4.pc'
       > to the PKG_CONFIG_PATH environment variable
       > Package 'libtiff-4', required by 'proj', not found
       > configure: PROJ: 9.4.1
       > checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no
       > configure: error: libproj or sqlite3 not found in standard or given locations.
       > ERROR: configuration failed for package 'terra'
       > * removing '/nix/store/nbwfdhv18l9c3k2zby6ni880x51py2gl-r-terra-1.7-78/library/terra'
       For full logs, run 'nix-store -l /nix/store/dza9i9zwxmnzzbwb0xchdksa9ari8dkw-r-terra-1.7-78.drv'.
error: build of '/nix/store/72afakncr7isb6c0s9n3fmq0976jvr4n-r-languageserver-0.3.16.drv', '/nix/store/dza9i9zwxmnzzbwb0xchdksa9ari8dkw-r-terra-1.7-78.drv' failed

Additional context

Build succeeds on Linux, and also succeeds on Darwin with following fix:

 terra = old.terra.overrideAttrs (attrs: {
                configureFlags = [
                    "--with-proj-lib=${pkgs.lib.getLib pkgs.proj}/lib"
                ];
            });

I PR'ed the fix, but it turns out that the issue is likely related to pkg-config, see, for context: https://github.com/NixOS/nixpkgs/pull/344498#pullrequestreview-2334701617

I don't have Apple hardware, so it's difficult for me to work on this.

Notify maintainers

pkg-config has no maintainers, so I'm pinging the last person to touch it @sevanspowell

Metadata

I can't run the below, because as explained above, I don't have the hardware.

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
output here

Add a :+1: reaction to issues you find important.

tomberek commented 2 days ago 
--- a/pkgs/development/libraries/proj/default.nix
+++ b/pkgs/development/libraries/proj/default.nix
@@ -49,6 +49,12 @@ stdenv.mkDerivation (finalAttrs: {
     "-include cstdint"
   ];

+  postPatch = ''
+    # These should be proper Requires, using the header needs their headers
+    substituteInPlace proj.pc.in \
+      --replace 'Requires.private' 'Requires'
+  '';
+

Might help.

saghirb commented 2 days ago

Metadata

I can't run the below, because as explained above, I don't have the hardware.

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
output here

Add a 👍 reaction to issues you find important.

% nix-shell -p nix-info --run "nix-info -m"
these 2 paths will be fetched (0.01 MiB download, 0.03 MiB unpacked):
  /nix/store/i17n43d9h64nzp5mjwd8zlj00i1zpab0-DarwinTools-1
  /nix/store/g16nnzwy639cmbhcggrwniba70fympah-nix-info
copying path '/nix/store/i17n43d9h64nzp5mjwd8zlj00i1zpab0-DarwinTools-1' from 'https://cache.nixos.org'...
copying path '/nix/store/g16nnzwy639cmbhcggrwniba70fympah-nix-info' from 'https://cache.nixos.org'...
 - system: `"x86_64-darwin"`
 - host os: `Darwin 21.6.0, macOS 10.16`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.24.8`
 - channels(root): `"nixpkgs"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixpkgs`
b-rodrigues commented 2 days ago 
+  postPatch = ''
+    # These should be proper Requires, using the header needs their headers
+    substituteInPlace proj.pc.in \
+      --replace 'Requires.private' 'Requires'
+  '';

Thanks for the pointer! Unfortunately that doesn’t seem to cut it. I’ve add the lines here: https://github.com/b-rodrigues/nixpkgs/blob/812fef336e2c1eacccba4e893e0338e65adb3e21/pkgs/development/libraries/proj/default.nix#L52

and tried to build the package from this branch, but still get the same error.

b-rodrigues commented 1 day ago

No sorry, that actually did the trick for rPackages.terra! It now builds successfully on linux and macOS with this patch. I thought it didn’t work because I didn’t read the error message carefully enough: what fails now is another package: rPackages.sf which is closely related to rPackages.terra (I was trying to build them all instead of individually, my bad). I thought that sf and terra might need the same fix as they seemed to have the same issue related to proj, but sf seems to require another fix to build successfully on darwin on top of the patch for proj:

checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no
configure: error: libproj or sqlite3 not found in standard or given locations.
ERROR: configuration failed for package ‘sf’
* removing ‘/nix/store/090rqzxk9dlm3001ksvsnsrp6mh6nsvq-r-sf-1.0-16/library/sf’
error: builder for '/nix/store/80gp841mb76m9rx208l4nxcmppkqpknc-r-sf-1.0-16.drv' failed with exit code 1;
       last 25 log lines:
       > checking whether the compiler supports GNU C... yes
       > checking whether clang accepts -g... yes
       > checking for clang option to enable C11 features... none needed
       > checking for stdio.h... yes
       > checking for stdlib.h... yes
       > checking for string.h... yes
       > checking for inttypes.h... yes
       > checking for stdint.h... yes
       > checking for strings.h... yes
       > checking for sys/stat.h... yes
       > checking for sys/types.h... yes
       > checking for unistd.h... yes
       > checking for gdal.h... yes
       > checking GDAL: linking with --libs only... yes
       > checking GDAL: /nix/store/vdl5cxnbm7n02kf9mn3417glarvik1r6-gdal-3.8.5/share/gdal/pcs.csv readable... no
       > checking GDAL: checking whether PROJ is available for linking:... yes
       > checking GDAL: checking whether PROJ is available for running:... yes
       > configure: GDAL: 3.8.5
       > configure: pkg-config proj exists, will use it
       > configure: using proj.h.
       > configure: PROJ: 9.4.1
       > checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no
       > configure: error: libproj or sqlite3 not found in standard or given locations.
       > ERROR: configuration failed for package ‘sf’
       > * removing ‘/nix/store/090rqzxk9dlm3001ksvsnsrp6mh6nsvq-r-sf-1.0-16/library/sf’
       For full logs, run 'nix log /nix/store/80gp841mb76m9rx208l4nxcmppkqpknc-r-sf-1.0-16.drv'.
error: 1 dependencies of derivation '/nix/store/fnd6sjisbfzambjmnpwd3pc602b68cnz-nix-shell.drv' failed to build

could it be something with sqlite3?

b-rodrigues commented 1 day ago

Scratch that, I think the culprit might be libgeotiff. Am trying the fix for proj on it as well: https://github.com/b-rodrigues/nixpkgs/blob/b1a7f21c1ecca85ac7dad0cc2f4d0f480f35968a/pkgs/development/libraries/libgeotiff/default.nix#L39

I’ll report the results once it’s done building.

b-rodrigues commented 1 day ago

Nope that wasn't it... so to summarize, for now the fix suggested by @tomberek works for rPackages.terra but rPackages.sf fails. Trying the same fix on libgeotiff, a dependency of rpackages.sf doesn't help