Failing build for {terra}

[joelnitta]

Please let me know if there is a better place to report this issue.

Attempt to build {terra} fails, apparently because of missing libtiff-4. Do I need to specify that manually (I thought the point of nix was that I don't need to worry about manually specifying deps)?

default.nix

# This file was generated by the {rix} R package v0.12.1 on 2024-09-25
# with following call:
# >rix::rix(r_ver = "189e5f171b163feb7791a9118afa778d9a1db81f",
#  > r_pkgs = "terra",
#  > system_pkgs = NULL,
#  > git_pkgs = NULL,
#  > ide = "code",
#  > project_path = ".",
#  > overwrite = TRUE)
# It uses nixpkgs' revision 189e5f171b163feb7791a9118afa778d9a1db81f for reproducibility purposes
# which will install R version latest.
# Report any issues to https://github.com/ropensci/rix
let
 pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/189e5f171b163feb7791a9118afa778d9a1db81f.tar.gz") {};

  rpkgs = builtins.attrValues {
    inherit (pkgs.rPackages) 
      languageserver
      terra;
  };

  system_packages = builtins.attrValues {
    inherit (pkgs) 
      glibcLocales
      nix
      R;
  };

in

pkgs.mkShell {
  LOCALE_ARCHIVE = if pkgs.system == "x86_64-linux" then "${pkgs.glibcLocales}/lib/locale/locale-archive" else "";
  LANG = "en_US.UTF-8";
   LC_ALL = "en_US.UTF-8";
   LC_TIME = "en_US.UTF-8";
   LC_MONETARY = "en_US.UTF-8";
   LC_PAPER = "en_US.UTF-8";
   LC_MEASUREMENT = "en_US.UTF-8";

  buildInputs = [  rpkgs  system_packages   ];

}

joelnitta ferns-distribution [jn] $ nix-shell
these 5 derivations will be built:
  /nix/store/hyl922p94nqqyjwkbdiq6hn817qkkib5-r-xmlparsedata-1.0.5.drv
  /nix/store/5fjbzdlha5grlyv0mblq2ksbw6wf1yc4-r-lintr-3.1.2.drv
  /nix/store/amspl2w42zrrhz55kli2n266kpkdiyj5-r-roxygen2-7.3.2.drv
  /nix/store/72afakncr7isb6c0s9n3fmq0976jvr4n-r-languageserver-0.3.16.drv
  /nix/store/dza9i9zwxmnzzbwb0xchdksa9ari8dkw-r-terra-1.7-78.drv
building '/nix/store/dza9i9zwxmnzzbwb0xchdksa9ari8dkw-r-terra-1.7-78.drv'...
Running phase: unpackPhase
unpacking source archive /nix/store/bl6pnawvyhfd7yv45s2vlchsj556n58h-terra_1.7-78.tar.gz
source root is terra
setting SOURCE_DATE_EPOCH to timestamp 1716388203 of file terra/MD5
Running phase: patchPhase
Running phase: configurePhase
Running phase: buildPhase
Running phase: checkPhase
Running phase: installPhase
During startup - Warning messages:
1: Setting LC_CTYPE failed, using "C" 
2: Setting LC_TIME failed, using "C" 
3: Setting LC_MESSAGES failed, using "C" 
4: Setting LC_MONETARY failed, using "C" 
* installing *source* package 'terra' ...
** package 'terra' successfully unpacked and MD5 sums checked
** using staged installation
configure: CC: clang
configure: CXX: /nix/store/bkhpl7q6fzghx08zzrmcckh12rrnnl29-clang-wrapper-16.0.6/bin/c++ -std=gnu++17
checking for gdal-config... /nix/store/zj7j809wk0c5iv5s7fsd32fnhh6185vr-gdal-3.9.2/bin/gdal-config
checking gdal-config usability... yes
configure: GDAL: 3.9.2
checking GDAL version >= 2.0.1... yes
checking for gcc... clang
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C... yes
checking whether clang accepts -g... yes
checking for clang option to enable C11 features... none needed
checking for stdio.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for strings.h... yes
checking for sys/stat.h... yes
checking for sys/types.h... yes
checking for unistd.h... yes
checking for gdal.h... yes
checking GDAL: linking with --libs only... yes
checking GDAL: /nix/store/zj7j809wk0c5iv5s7fsd32fnhh6185vr-gdal-3.9.2/share/gdal/pcs.csv readable... no
checking GDAL: checking whether PROJ is available for linking:... yes
checking GDAL: checking whether PROJ is available for running:... yes
configure: GDAL: 3.9.2
configure: pkg-config proj exists, will use it
configure: using proj.h.
Package libtiff-4 was not found in the pkg-config search path.
Perhaps you should add the directory containing `libtiff-4.pc'
to the PKG_CONFIG_PATH environment variable
Package 'libtiff-4', required by 'proj', not found
configure: PROJ: 9.4.1
checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no
configure: error: libproj or sqlite3 not found in standard or given locations.
ERROR: configuration failed for package 'terra'
* removing '/nix/store/nbwfdhv18l9c3k2zby6ni880x51py2gl-r-terra-1.7-78/library/terra'
error: builder for '/nix/store/dza9i9zwxmnzzbwb0xchdksa9ari8dkw-r-terra-1.7-78.drv' failed with exit code 1;
       last 25 log lines:
       > checking for stdlib.h... yes
       > checking for string.h... yes
       > checking for inttypes.h... yes
       > checking for stdint.h... yes
       > checking for strings.h... yes
       > checking for sys/stat.h... yes
       > checking for sys/types.h... yes
       > checking for unistd.h... yes
       > checking for gdal.h... yes
       > checking GDAL: linking with --libs only... yes
       > checking GDAL: /nix/store/zj7j809wk0c5iv5s7fsd32fnhh6185vr-gdal-3.9.2/share/gdal/pcs.csv readable... no
       > checking GDAL: checking whether PROJ is available for linking:... yes
       > checking GDAL: checking whether PROJ is available for running:... yes
       > configure: GDAL: 3.9.2
       > configure: pkg-config proj exists, will use it
       > configure: using proj.h.
       > Package libtiff-4 was not found in the pkg-config search path.
       > Perhaps you should add the directory containing `libtiff-4.pc'
       > to the PKG_CONFIG_PATH environment variable
       > Package 'libtiff-4', required by 'proj', not found
       > configure: PROJ: 9.4.1
       > checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no
       > configure: error: libproj or sqlite3 not found in standard or given locations.
       > ERROR: configuration failed for package 'terra'
       > * removing '/nix/store/nbwfdhv18l9c3k2zby6ni880x51py2gl-r-terra-1.7-78/library/terra'
       For full logs, run 'nix-store -l /nix/store/dza9i9zwxmnzzbwb0xchdksa9ari8dkw-r-terra-1.7-78.drv'.
error: build of '/nix/store/72afakncr7isb6c0s9n3fmq0976jvr4n-r-languageserver-0.3.16.drv', '/nix/store/dza9i9zwxmnzzbwb0xchdksa9ari8dkw-r-terra-1.7-78.drv' failed

[b-rodrigues]

Are you on macOS? Some packages can fail sometimes as they get updated and their requirements evolve. macOS packages don’t get built on Hydra (CI platform for NixOS) afaik, which means some can slip through the cracks. Looking at terra’s definition on nixpkgs, https://github.com/NixOS/nixpkgs/blob/437cd5436f17b51f6ebffa15f5d9b20eeaf0f4cc/pkgs/development/r-modules/default.nix#L663 libtiff is indeed missing. The fix would be for us to add it.

I was able to build it and use in a temporary shell on Linux:

 nix-shell  -p R rPackages.terra

so I guess this requirement is macOS-specific. We will open a PR to add it and it should then work.

[b-rodrigues]

@philipp-baumann can you reproduce the error on your mac? Does adding libtiff4 to the nativeBuildInputs of terra fix the build?

EDIT:

I knew this error ringed a bell, because I was confronted to this issue for the rstats-on-nix/nixpkgs fork. This was the fix I used at the time, but that never made it upstream:

 terra = old.terra.overrideAttrs (attrs: {
                configureFlags = [
                    "--with-proj-lib=${pkgs.lib.getLib pkgs.proj}/lib"
                ];
            });

From my limited testing, this is needed for macOS. If that turns out to be the case, then @philipp-baumann we could open a PR upstream to fix these packages. There’s others like terra, such as vapour, that require the same fix. They’re listed here: https://github.com/rstats-on-nix/nixpkgs/blob/0dac872330b0b712ba830b36c399eac66741773a/.github/workflows/r-daily.yml#L62

[b-rodrigues]

Some testing on Github Actions seems to confirm this is a macOS-specific issue:

[b-rodrigues]

could you try with this expression

let
 pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/189e5f171b163feb7791a9118afa778d9a1db81f.tar.gz") {};

 terra = pkgs.rPackages.terra.overrideAttrs (attrs: {
                configureFlags = [
                    "--with-proj-lib=${pkgs.lib.getLib pkgs.proj}/lib"
                ];
            });

  rpkgs = builtins.attrValues {
    inherit (pkgs.rPackages) 
      languageserver;
  };

  system_packages = builtins.attrValues {
    inherit (pkgs) 
      glibcLocales
      nix
      R;
  };

in

pkgs.mkShell {
  LOCALE_ARCHIVE = if pkgs.system == "x86_64-linux" then "${pkgs.glibcLocales}/lib/locale/locale-archive" else "";
  LANG = "en_US.UTF-8";
   LC_ALL = "en_US.UTF-8";
   LC_TIME = "en_US.UTF-8";
   LC_MONETARY = "en_US.UTF-8";
   LC_PAPER = "en_US.UTF-8";
   LC_MEASUREMENT = "en_US.UTF-8";

  buildInputs = [  rpkgs  system_packages  terra  ];

}

I have overriden the definition of terra to make it work for macOS. If this works, we are going to propose this fix upstream so future users (and future you) don’t need to override the expression anymore.

[philipp-baumann]

could you try with this expression

let
 pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/189e5f171b163feb7791a9118afa778d9a1db81f.tar.gz") {};

 terra = pkgs.rPackages.terra.overrideAttrs (attrs: {
                configureFlags = [
                    "--with-proj-lib=${pkgs.lib.getLib pkgs.proj}/lib"
                ];
            });

  rpkgs = builtins.attrValues {
    inherit (pkgs.rPackages) 
      languageserver;
  };

  system_packages = builtins.attrValues {
    inherit (pkgs) 
      glibcLocales
      nix
      R;
  };

in

pkgs.mkShell {
  LOCALE_ARCHIVE = if pkgs.system == "x86_64-linux" then "${pkgs.glibcLocales}/lib/locale/locale-archive" else "";
  LANG = "en_US.UTF-8";
   LC_ALL = "en_US.UTF-8";
   LC_TIME = "en_US.UTF-8";
   LC_MONETARY = "en_US.UTF-8";
   LC_PAPER = "en_US.UTF-8";
   LC_MEASUREMENT = "en_US.UTF-8";

  buildInputs = [  rpkgs  system_packages  terra  ];

}

I have overriden the definition of terra to make it work for macOS. If this works, we are going to propose this fix upstream so future users (and future you) don’t need to override the expression anymore.

Nice, thanks! This builds for me:

> library("terra")
terra 1.7.78
> sessionInfo()
R version 4.4.1 (2024-06-14)
Platform: aarch64-apple-darwin23.6.0
Running under: macOS Sonoma 14.2.1

Matrix products: default
BLAS:   /nix/store/p0piq9717s687n34sarwn07wzbimmvig-blas-3/lib/libblas.dylib 
LAPACK: /nix/store/liz9lfphmcygp4v1n33mav5gspi8cnfl-openblas-0.3.28/lib/libopenblasp-r0.3.28.dylib;  LAPACK version 3.12.0

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: Europe/Zurich
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] terra_1.7-78

loaded via a namespace (and not attached):
[1] compiler_4.4.1   Rcpp_1.0.13      codetools_0.2-20
> .libPaths()
 [1] "/nix/store/czybyqzscyc5jy321jpvg3p0ss2v9if6-r-languageserver-0.3.16/library"              
 [2] "/nix/store/wzq9mscja7ifz8vij2yn62a574rx5b7p-r-R6-2.5.1/library"                           
 [3] "/nix/store/bnbdjp9lgqfby7p6pmcc0f2swf34lazq-r-callr-3.7.6/library"                        
 [4] "/nix/store/2y4msn8zpa42bxki3c8x445amwzss1yj-r-processx-3.8.4/library"                     
 [5] "/nix/store/s5ln693yvwa4fnj4d2ldk8whwvn9ri2b-r-ps-1.7.7/library"                           
 [6] "/nix/store/8rx9y42gj6igbd0nz4qhbr6kj5j22s1c-r-collections-0.3.7/library"                  
 [7] "/nix/store/91hayx7v9qissjgnv3ckzp3a3z8ivnzy-r-fs-1.6.4/library"                           
 [8] "/nix/store/0858j12dp73gn36nyv2mgnbvv03p0mic-r-jsonlite-1.8.8/library"                     
 [9] "/nix/store/fkwip5m44wpd1p0arii0hva4lcgki4vj-r-lintr-3.1.2/library"                        
[10] "/nix/store/831zdnhqqdfazqqbgl4mvk7lryb3jym9-r-backports-1.5.0/library"                    
[11] "/nix/store/1hq4iys5gfa8wgsv1nbapbp0wsljf47j-r-codetools-0.2-20/library"                   
[12] "/nix/store/zkhqp7icmj8qfpi65kgzv0wr1al5gqgx-r-cyclocomp-1.1.1/library"                    
[13] "/nix/store/6a5856gaaqgbx77qp7gcj7sh1v3c3vgx-r-crayon-1.5.3/library"                       
[14] "/nix/store/msiwzrpvpfzc51c5b0wgklh3r0pvbcaf-r-desc-1.4.3/library"                         
[15] "/nix/store/2r1qnz2k8gh91kqpyz4zrzyk6hhbdwhz-r-cli-3.6.3/library"                          
[16] "/nix/store/byfk2qs231gv00916fzaqk1j7iy2b3wy-r-remotes-2.5.0/library"                      
[17] "/nix/store/grpd38ms14jbc4cinwxcfw7fggn4jxib-r-withr-3.0.1/library"                        
[18] "/nix/store/4xmndw1khj9bdjli3045wpfn5gfq7p0k-r-digest-0.6.36/library"                      
[19] "/nix/store/n0xnlaqxmyvlcacsc2yghmk0rjpbsrsz-r-glue-1.7.0/library"                         
[20] "/nix/store/49jwbgcnncp52mswi1ipg512jxhy815f-r-knitr-1.48/library"                         
[21] "/nix/store/vay2qxyaq3m825wds48s842jdqzprjgv-r-evaluate-0.24.0/library"                    
[22] "/nix/store/j1h0448vmgf1lk6ammw4cv15gxf3zp2v-r-highr-0.11/library"                         
[23] "/nix/store/mf1i9ca06c8azjxjj400mbjh1iyk3p67-r-xfun-0.47/library"                          
[24] "/nix/store/p0gngi6wac6j36aclpz0bkwri9kz1yjy-r-yaml-2.3.10/library"                        
[25] "/nix/store/rxg8sq9ggd6nbgfl806nhvpj9zmcagyc-r-rex-1.2.1/library"                          
[26] "/nix/store/v4pjdah6834m3kgpa18zlimsjl7g84jc-r-lazyeval-0.2.2/library"                     
[27] "/nix/store/z4x0isw6xs17la7ylscphwccxs2qb3wy-r-xml2-1.3.6/library"                         
[28] "/nix/store/l3gpy7b266lk87414jgb2qlxb3cf5cji-r-rlang-1.1.4/library"                        
[29] "/nix/store/67912y9ijyzp83n1fyv3i00l8ic5sq4y-r-xmlparsedata-1.0.5/library"                 
[30] "/nix/store/yi14f9hdw2clmsjfricp4ml87l1n6byw-r-roxygen2-7.3.2/library"                     
[31] "/nix/store/dg8fyqwmx61kxrs3zmllna4afwsfmya3-r-brew-1.0-10/library"                        
[32] "/nix/store/j1jib2jzc34a8zhx31hmswgiajwm70s7-r-commonmark-1.9.1/library"                   
[33] "/nix/store/8qgf4h83855fqxvggbg3gsjs9vbv5hl7-r-cpp11-0.4.7/library"                        
[34] "/nix/store/n36byxzh82xhgngsvi5vg4qlla84vx0w-r-pkgload-1.4.0/library"                      
[35] "/nix/store/by7pxkr9ziz1i97ib9iq9g7r4vz60cxq-r-lifecycle-1.0.4/library"                    
[36] "/nix/store/dj31rnrrprzb1f7hp8qsxiyvjw2y0776-r-pkgbuild-1.4.4/library"                     
[37] "/nix/store/smv6flkhmji0a54l5j8s4353z4knk16x-r-rprojroot-2.0.4/library"                    
[38] "/nix/store/zal3y264fzcj4hb05bc03nb0d854k4z2-r-purrr-1.0.2/library"                        
[39] "/nix/store/9bja5awlgvha85ff114yvnirkm43q68v-r-magrittr-2.0.3/library"                     
[40] "/nix/store/ysmdhjpbzslba5q6wz41a06p1a7xinki-r-vctrs-0.6.5/library"                        
[41] "/nix/store/qv69qnqsgkhkjw04gc8jkbmk9a3rwkj6-r-stringi-1.8.4/library"                      
[42] "/nix/store/887nz9yjcbr7brlq7hjak9fx7rwmk7gq-r-stringr-1.5.1/library"                      
[43] "/nix/store/1mpzgjshjalkcqysdbpg8ya5xh018dxf-r-styler-1.10.3/library"                      
[44] "/nix/store/n7snbv37c7rrz8jfksri6hs0wamhpxas-r-R.cache-0.16.0/library"                     
[45] "/nix/store/5skyfshvs5mrkkszss05dvxj62xs3w6v-r-R.methodsS3-1.8.2/library"                  
[46] "/nix/store/q5hvv3cf0lpvprcs55mz0d6cyhiq6cmr-r-R.oo-1.26.0/library"                        
[47] "/nix/store/fkr7mgkzf5afzxacdxyrx53z3zd950ad-r-R.utils-2.12.3/library"                     
[48] "/nix/store/l876hksjiz1240d5in5ksdzskag0w15k-nix-2.18.7/Library"                           
[49] "/nix/store/by00zpyy1kvgkxdm3wbi1sirh3831ik5-r-terra-1.7-78/library"                       
[50] "/nix/store/ix1w8f10h1mpc9nhamrj2fdni4gks4ll-r-Rcpp-1.0.13/library"                        
[51] "/nix/store/kcn9w36a573nhkbl5ci7aa2vdk2q5k16-apple-framework-CoreFoundation-11.0.0/Library"
[52] "/nix/store/l5hm8k0r79bk1j2556wp1l4cmmrvza7v-R-4.4.1/lib/R/library" 

[philipp-baumann]

Are you on macOS? Some packages can fail sometimes as they get updated and their requirements evolve. macOS packages don’t get built on Hydra (CI platform for NixOS) afaik, which means some can slip through the cracks. Looking at terra’s definition on nixpkgs, https://github.com/NixOS/nixpkgs/blob/437cd5436f17b51f6ebffa15f5d9b20eeaf0f4cc/pkgs/development/r-modules/default.nix#L663 libtiff is indeed missing. The fix would be for us to add it.

I was able to build it and use in a temporary shell on Linux:

 nix-shell  -p R rPackages.terra

so I guess this requirement is macOS-specific. We will open a PR to add it and it should then work.

very much agree :-) the hydra test farm should indeed be activated for darwin in the r-module of nixpkgs. @b-rodrigues Would be nice if you make a PR there with the fix, i can confirm it works there for darwin in PR.

[joelnitta]

@b-rodrigues , @philipp-baumann thanks for your prompt troubleshooting.

I am indeed on a Mac. (Intel processor, Sonoma OS)

The R command used to generate default.nix:

rix::rix(
  r_ver = "latest",
  r_pkgs = "terra",
  system_pkgs = NULL,
  git_pkgs = NULL,
  ide = "code",
  project_path = ".",
  overwrite = TRUE
)

I can also confirm the fix above works on my machine.

[b-rodrigues]

I have also opened a PR https://github.com/NixOS/nixpkgs/pull/344498 to fix this upstream

[joelnitta]

@b-rodrigues I also need to install {sf}, which I see you included in your PR. Could you let me know the temporary fix to default.nix that would allow me to use it until that PR gets merged?

[b-rodrigues]

let
 pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/189e5f171b163feb7791a9118afa778d9a1db81f.tar.gz") {};

 terra = pkgs.rPackages.terra.overrideAttrs (attrs: {
                configureFlags = [
                    "--with-proj-lib=${pkgs.lib.getLib pkgs.proj}/lib"
                ];
            });

sf = pkgs.rPackages.sf.overrideAttrs (attrs: {
                configureFlags = [
                    "--with-proj-lib=${pkgs.lib.getLib pkgs.proj}/lib"
                ];
            });

  rpkgs = builtins.attrValues {
    inherit (pkgs.rPackages) 
      languageserver;
  };

  system_packages = builtins.attrValues {
    inherit (pkgs) 
      glibcLocales
      nix
      R;
  };

in

pkgs.mkShell {
  LOCALE_ARCHIVE = if pkgs.system == "x86_64-linux" then "${pkgs.glibcLocales}/lib/locale/locale-archive" else "";
  LANG = "en_US.UTF-8";
   LC_ALL = "en_US.UTF-8";
   LC_TIME = "en_US.UTF-8";
   LC_MONETARY = "en_US.UTF-8";
   LC_PAPER = "en_US.UTF-8";
   LC_MEASUREMENT = "en_US.UTF-8";

  buildInputs = [  rpkgs  system_packages  terra sf ];

} 

[b-rodrigues]

There are currently many packages broken due to an update to libxml2 that deprecated a feature... hopefully you don't need such a package because for now we don't have a fix, other than using an older nixpkgs revision... sf should be fine though

[joelnitta]

I'm trying to incorporate your code into a longer list of packages generated with rix:

let
 pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/189e5f171b163feb7791a9118afa778d9a1db81f.tar.gz") {};

 terra = pkgs.rPackages.terra.overrideAttrs (attrs: {
                configureFlags = [
                    "--with-proj-lib=${pkgs.lib.getLib pkgs.proj}/lib"
                ];
            });

sf = pkgs.rPackages.sf.overrideAttrs (attrs: {
                configureFlags = [
                    "--with-proj-lib=${pkgs.lib.getLib pkgs.proj}/lib"
                ];
            });

  rpkgs = builtins.attrValues {
    inherit (pkgs.rPackages) 
      data_table
      dplyr
      exactextractr
      languageserver
      mapdata
      raster
      rnaturalearth
      rnaturalearthdata
      tidyverse;
  };

  git_archive_pkgs = [
    (pkgs.rPackages.buildRPackage {
      name = "rnaturalearthhires";
      src = pkgs.fetchgit {
        url = "https://github.com/ropensci/rnaturalearthhires/";
        rev = "e02c28dab09c511cbe4b1da429ab69c001c2d70c";
        sha256 = "sha256-0pjPy8zR1XLfk1ooVgqO1eYxDLeEiLxLyDkMVVpqRos=";
      };
      propagatedBuildInputs = builtins.attrValues {
        inherit (pkgs.rPackages) ;
      };
    })
   ];

  system_packages = builtins.attrValues {
    inherit (pkgs) 
      glibcLocales
      nix
      R;
  };

in

pkgs.mkShell {
  LOCALE_ARCHIVE = if pkgs.system == "x86_64-linux" then "${pkgs.glibcLocales}/lib/locale/locale-archive" else "";
  LANG = "en_US.UTF-8";
   LC_ALL = "en_US.UTF-8";
   LC_TIME = "en_US.UTF-8";
   LC_MONETARY = "en_US.UTF-8";
   LC_PAPER = "en_US.UTF-8";
   LC_MEASUREMENT = "en_US.UTF-8";

  buildInputs = [ git_archive_pkgs rpkgs  system_packages   ];

}

But it's not working.

joelnitta ferns-distribution [jn] $ nix-shell                      
these 21 derivations will be built:
  /nix/store/jrab3hgac7sifh2d98g80qnxkmhbyfm1-r-textshaping-0.4.0.drv
  /nix/store/0br03lcqgw6daxwkg7gpg6v1vndpsnik-r-ragg-1.3.2.drv
  /nix/store/b8j4c19lmsvd9zxj8ccqfhf7c0kjcmfq-r-sf-1.0-16.drv
  /nix/store/dza9i9zwxmnzzbwb0xchdksa9ari8dkw-r-terra-1.7-78.drv
  /nix/store/d7527hsq588v6gc4mq3av5klwp27k35g-r-raster-3.6-26.drv
  /nix/store/0q8vipk7ww5i2h1g71jf15li690anq57-r-exactextractr-0.10.0.drv
  /nix/store/mnhvzvhh3q884bn65di6yi3ri7j3ywnc-r-tinytex-0.52.drv
  /nix/store/s37y487xsxwxhfl2m4qx8i6a57z9hn56-r-rmarkdown-2.28.drv
  /nix/store/1hzj7qll4167k0rlx4jqsj6sl16q8scc-r-reprex-2.1.1.drv
  /nix/store/2h7cir33vkgqa741cni9jxj3qzvyi3cn-r-tzdb-0.4.0.drv
  /nix/store/9c3xki7gy240n21z8yz3skbkd2iywkg5-r-uuid-1.2-1.drv
  /nix/store/bpgkcsxg5idyk3qifxyxwv0y1lddi0b8-r-googledrive-2.1.1.drv
  /nix/store/mm0g90c7bl9miikbf1z0d2csaqgq67sr-r-rnaturalearth-1.0.1.drv
  /nix/store/n6kbjpkpgh6280rf4yh7amin9b6rd8d9-r-ids-1.0.1.drv
  /nix/store/xp463cpxlynfbd7ls2bgjhg77rc9pab5-r-timechange-0.3.0.drv
  /nix/store/p7x7bl73p2r0r4bwhfpm2hcfgikaiqq7-r-lubridate-1.9.3.drv
  /nix/store/v3i7gr1l59z2hmj3i8rzacd56jjp7sdq-r-vroom-1.6.5.drv
  /nix/store/rp6cb148xcsycnqij1zddacfvk35nhs3-r-readr-2.1.5.drv
  /nix/store/w1n2v0pkl93b4lf2sn3h8fan8lb8iglk-r-googlesheets4-1.1.1.drv
  /nix/store/xplc2da4fkpmj8z4j1g45h5xx4mcg41p-r-haven-2.5.4.drv
  /nix/store/p501fkj21bwcdvnyfvznps94pwbwxj8v-r-tidyverse-2.0.0.drv
building '/nix/store/b8j4c19lmsvd9zxj8ccqfhf7c0kjcmfq-r-sf-1.0-16.drv'...
Running phase: unpackPhase
unpacking source archive /nix/store/50946lkz5s1vr0bpmbj8qs52hpk6iyw4-sf_1.0-16.tar.gz
source root is sf
setting SOURCE_DATE_EPOCH to timestamp 1711324203 of file sf/MD5
Running phase: patchPhase
Running phase: configurePhase
Running phase: buildPhase
Running phase: checkPhase
Running phase: installPhase
During startup - Warning messages:
1: Setting LC_CTYPE failed, using "C" 
2: Setting LC_TIME failed, using "C" 
3: Setting LC_MESSAGES failed, using "C" 
4: Setting LC_MONETARY failed, using "C" 
* installing *source* package 'sf' ...
** package 'sf' successfully unpacked and MD5 sums checked
** using staged installation
configure: CC: clang
configure: CXX: /nix/store/bkhpl7q6fzghx08zzrmcckh12rrnnl29-clang-wrapper-16.0.6/bin/c++ -std=gnu++17
checking for gdal-config... /nix/store/zj7j809wk0c5iv5s7fsd32fnhh6185vr-gdal-3.9.2/bin/gdal-config
checking gdal-config usability... yes
configure: GDAL: 3.9.2
checking GDAL version >= 2.0.1... yes
checking for gcc... clang
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether the compiler supports GNU C... yes
checking whether clang accepts -g... yes
checking for clang option to enable C11 features... none needed
checking for stdio.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for strings.h... yes
checking for sys/stat.h... yes
checking for sys/types.h... yes
checking for unistd.h... yes
checking for gdal.h... yes
checking GDAL: linking with --libs only... yes
checking GDAL: /nix/store/zj7j809wk0c5iv5s7fsd32fnhh6185vr-gdal-3.9.2/share/gdal/pcs.csv readable... no
checking GDAL: checking whether PROJ is available for linking:... yes
checking GDAL: checking whether PROJ is available for running:... yes
configure: GDAL: 3.9.2
configure: pkg-config proj exists, will use it
configure: using proj.h.
Package Lerc was not found in the pkg-config search path.
Perhaps you should add the directory containing `Lerc.pc'
to the PKG_CONFIG_PATH environment variable
Package 'Lerc', required by 'libtiff-4', not found
configure: PROJ: 9.4.1
checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no
configure: error: libproj or sqlite3 not found in standard or given locations.
ERROR: configuration failed for package 'sf'
* removing '/nix/store/20d75fx1vacfaywnd3q7xd8j6dy4jp5m-r-sf-1.0-16/library/sf'
error: builder for '/nix/store/b8j4c19lmsvd9zxj8ccqfhf7c0kjcmfq-r-sf-1.0-16.drv' failed with exit code 1;
       last 25 log lines:
       > checking for stdlib.h... yes
       > checking for string.h... yes
       > checking for inttypes.h... yes
       > checking for stdint.h... yes
       > checking for strings.h... yes
       > checking for sys/stat.h... yes
       > checking for sys/types.h... yes
       > checking for unistd.h... yes
       > checking for gdal.h... yes
       > checking GDAL: linking with --libs only... yes
       > checking GDAL: /nix/store/zj7j809wk0c5iv5s7fsd32fnhh6185vr-gdal-3.9.2/share/gdal/pcs.csv readable... no
       > checking GDAL: checking whether PROJ is available for linking:... yes
       > checking GDAL: checking whether PROJ is available for running:... yes
       > configure: GDAL: 3.9.2
       > configure: pkg-config proj exists, will use it
       > configure: using proj.h.
       > Package Lerc was not found in the pkg-config search path.
       > Perhaps you should add the directory containing `Lerc.pc'
       > to the PKG_CONFIG_PATH environment variable
       > Package 'Lerc', required by 'libtiff-4', not found
       > configure: PROJ: 9.4.1
       > checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no
       > configure: error: libproj or sqlite3 not found in standard or given locations.
       > ERROR: configuration failed for package 'sf'
       > * removing '/nix/store/20d75fx1vacfaywnd3q7xd8j6dy4jp5m-r-sf-1.0-16/library/sf'
       For full logs, run 'nix-store -l /nix/store/b8j4c19lmsvd9zxj8ccqfhf7c0kjcmfq-r-sf-1.0-16.drv'.
error: 1 dependencies of derivation '/nix/store/0q8vipk7ww5i2h1g71jf15li690anq57-r-exactextractr-0.10.0.drv' failed to build
error: build of '/nix/store/0q8vipk7ww5i2h1g71jf15li690anq57-r-exactextractr-0.10.0.drv', '/nix/store/d7527hsq588v6gc4mq3av5klwp27k35g-r-raster-3.6-26.drv', '/nix/store/mm0g90c7bl9miikbf1z0d2csaqgq67sr-r-rnaturalearth-1.0.1.drv', '/nix/store/p501fkj21bwcdvnyfvznps94pwbwxj8v-r-tidyverse-2.0.0.drv' failed

Any suggestions?

[philipp-baumann]

I have also opened a PR NixOS/nixpkgs#344498 to fix this upstream

I just reviewed there. @joelnitta Since you have X86-64 (Intel) and I have a AArch64 , you could complete the review in the PR like this and reporting the output like I did, that would be awesome, if you have time :-). You can review like this

# clone nixpkgks
# pull Bruno's PR
git fetch https://github.com/NixOS/nixpkgs.git pull/344498/head:pr-344498
git checkout pr-344498
# Test build on latest PR branch state
nix-shell -I nixpkgs=. -p rPackages.terra R

• show the R output at PR checkout
• show the output of nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"

Cheers

[b-rodrigues]

Perhaps the simplest would be to generate the expression and then replace

pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/189e5f171b163feb7791a9118afa778d9a1db81f.tar.gz") {};

with

pkgs = import (fetchTarball "https://github.com/b-rodrigues/nixpkgs/archive/3b2590eee2b057ada39066e536c3a0ebe66d2f8d.tar.gz") {};

this will be using my fork with the fix until it gets merged upstream, no need to override definitions in this case

[joelnitta]

Thanks, I'll try that.

Might be nice to have pkgs as an additional argument to rix::rix() for this sort of thing...

[b-rodrigues]

That could indeed become an option

[joelnitta]

@philipp-baumann

I followed your instructions, but the R output looks a little off

joelnitta nixpkgs [pr-344498] $ nix-shell -I nixpkgs=. -p rPackages.terra R

[nix-shell:~/repos/nixpkgs]$ R

R version 4.4.1 (2024-06-14) -- "Race for Your Life"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin23.6.0

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

During startup - Warning messages:
1: Setting LC_CTYPE failed, using "C" 
2: Setting LC_COLLATE failed, using "C" 
3: Setting LC_TIME failed, using "C" 
4: Setting LC_MESSAGES failed, using "C" 
5: Setting LC_MONETARY failed, using "C" 
> library(terra)
code for methods in class "Rcpp_SpatCategories" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatCategories" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatDataFrame" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatDataFrame" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatExtent" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatExtent" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatFactor" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatFactor" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatGraph" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatGraph" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatMessages" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatMessages" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatOptions" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatOptions" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatRaster" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatRaster" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatRasterCollection" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatRasterCollection" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatRasterStack" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatRasterStack" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatSRS" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatSRS" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatTime_v" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatTime_v" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatVector" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatVector" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatVectorCollection" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatVectorCollection" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatVectorProxy" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatVectorProxy" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
terra 1.7.78
> 

[b-rodrigues]

BTW, I tested my suggestion and got it working on Linux, my guess is that with my fix it should work on macOS as well:

let
pkgs = import (fetchTarball "https://github.com/b-rodrigues/nixpkgs/archive/3b2590eee2b057ada39066e536c3a0ebe66d2f8d.tar.gz") {};

  rpkgs = builtins.attrValues {
    inherit (pkgs.rPackages) 
      data_table
      dplyr
      exactextractr
      languageserver
      mapdata
      sf
      terra
      raster
      rnaturalearth
      rnaturalearthdata
      tidyverse;
  };

  git_archive_pkgs = [
    (pkgs.rPackages.buildRPackage {
      name = "rnaturalearthhires";
      src = pkgs.fetchgit {
        url = "https://github.com/ropensci/rnaturalearthhires/";
        rev = "e02c28dab09c511cbe4b1da429ab69c001c2d70c";
        sha256 = "sha256-0pjPy8zR1XLfk1ooVgqO1eYxDLeEiLxLyDkMVVpqRos=";
      };
      propagatedBuildInputs = builtins.attrValues {
        inherit (pkgs.rPackages) ;
      };
    })
   ];

  system_packages = builtins.attrValues {
    inherit (pkgs) 
      glibcLocales
      nix
      R;
  };

in

pkgs.mkShell {
  LOCALE_ARCHIVE = if pkgs.system == "x86_64-linux" then "${pkgs.glibcLocales}/lib/locale/locale-archive" else "";
  LANG = "en_US.UTF-8";
   LC_ALL = "en_US.UTF-8";
   LC_TIME = "en_US.UTF-8";
   LC_MONETARY = "en_US.UTF-8";
   LC_PAPER = "en_US.UTF-8";
   LC_MEASUREMENT = "en_US.UTF-8";

  buildInputs = [ git_archive_pkgs rpkgs  system_packages   ];

}

You'll notice I use my fork with the fix, and added sf and terra back to the list of R packages. Sorry your first experience with Nix wasn't as smooth as it could have been, but unfortunately we have limited capacity when it comes to building and testing on macOS, so these kind of issues are really helpful for us!

[joelnitta]

Yes, I can confirm that setting pkgs = import (fetchTarball "https://github.com/b-rodrigues/nixpkgs/archive/3b2590eee2b057ada39066e536c3a0ebe66d2f8d.tar.gz") {}; works for me as well.

No problem, and thanks again for providing a solution.

My impression so far is that (ahem when it works 😉) nix seems pretty great, especially with handling the system library dependencies of R packages. The project where I'm trying this is one that picks up a collaborator's code with a bunch of library() calls, so it's pretty awesome to be able to just pop all those package names into rix() and have an environment up and running.

[philipp-baumann]

@philipp-baumann

I followed your instructions, but the R output looks a little off

joelnitta nixpkgs [pr-344498] $ nix-shell -I nixpkgs=. -p rPackages.terra R

[nix-shell:~/repos/nixpkgs]$ R

R version 4.4.1 (2024-06-14) -- "Race for Your Life"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin23.6.0

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

During startup - Warning messages:
1: Setting LC_CTYPE failed, using "C" 
2: Setting LC_COLLATE failed, using "C" 
3: Setting LC_TIME failed, using "C" 
4: Setting LC_MESSAGES failed, using "C" 
5: Setting LC_MONETARY failed, using "C" 
> library(terra)
code for methods in class "Rcpp_SpatCategories" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatCategories" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatDataFrame" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatDataFrame" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatExtent" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatExtent" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatFactor" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatFactor" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatGraph" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatGraph" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatMessages" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatMessages" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatOptions" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatOptions" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatRaster" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatRaster" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatRasterCollection" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatRasterCollection" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatRasterStack" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatRasterStack" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatSRS" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatSRS" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatTime_v" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatTime_v" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatVector" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatVector" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatVectorCollection" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatVectorCollection" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatVectorProxy" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
code for methods in class "Rcpp_SpatVectorProxy" was not checked for suspicious field assignments (recommended package 'codetools' not available?)
terra 1.7.78
> 

yeah i can confirm this codetools hickup, to be honest i can't explain this, maybe something to ask in terra, RCpp?

[jrosell]

There are currently many packages broken due to an update to libxml2 that deprecated a feature... hopefully you don't need such a package because for now we don't have a fix, other than using an older nixpkgs revision... sf should be fine though

Could one stick to one specific R package version (r_pkgs) and be ok using nix? I guess using git_pkgs could be another option.

[b-rodrigues]

what do you mean?

[jrosell]

For example, try to add versions just like r_pkgs = c("terra@1.6-53") or r_pkgs = c("terra@1.6")

[b-rodrigues]

This is risky because what happens when you do this, is that the packages get installed by being compiled using your system libraries, and so it somewhat defeats the purposes of using Nix. As described in the documentation, we don’t recommend using this, and provide this as a facility that can be useful in certain situations, especially if one absolutely needs a specific version of a package.

Currently, because of this libxml2 issue on nixpkgs, we recommend using a pinned version of R instead of setting r_ver = "latest". This way, the dev shell you end up building is completely coherent.

[b-rodrigues]

@joelnitta can we close this?

[joelnitta]

I'm OK either way @b-rodrigues. We have a temporary fix thanks to the solution using your fork. But the PR at nixpkgs is currently blocked pending some requested changes.

[b-rodrigues]

Yep, the fix seems to be with pkg-config and not with the packages themselves. Unfortunately this will be difficult for me to fix, as I don’t have the hardware, nor familiarity with how pkg-config works.

[joelnitta]

Like I said, your call: clearly it's outside the scope of the package (TBH that's why I wasn't sure about posting here in the first place, but didn't know where else to go), but {terra} is pretty popular for geospatial analysis with R so I can imagine other {rix} users running into this.

[b-rodrigues]

I opened an issue : https://github.com/NixOS/nixpkgs/issues/346425 to see if we can get help for this

[b-rodrigues]

The PR has been merged! thanks for raising the issue @joelnitta !

[joelnitta]

Great! To follow up, what do I need to do to implement this when writing default.nix with rix()?

[b-rodrigues]

The PR has been merged to master, so any very recent hash from master will include it. Use this for example in rix() as the r_ver "97f0c067f3d10b7dff87881de8bbc9710ab8a51e".

Using "latest" gets the latest commit from the unstable branch, which lags some days behind master, so it won't include the fix yet.

If you want to use an older version of R, you'd need to fix the packages as shown here by overriding their definitions.

I'm thinking of doing a fork of nixpkgs with 5 years of CRAN packages with all fixes backported, I have a pretty good idea of how to achieve this.

[joelnitta]

Got it, thanks.

So in the docs for rix(), where is says "For reproducibility purposes, you can also provide a nixpkgs revision directly," "revision" means "commit"? For new users, it would probably help to have a link to the repo there too.