Compilation on MacOS Monterey M1 (ARM64 architecture), using Homebrew

[aourednik]

Same problem as here: https://github.com/r-spatial/sf/issues/1848 , now solved for the sf package.

Building has trouble with new location of gdal-config, libproj and sqlite when using Homebrew for ARM64 (instead of usr/local/ , homebrew intalls components to the arm64-compatible directory /opt/homebrew/opt/)

install.packages("terra",configure.args = "--with-gdal-config=/opt/homebrew/opt/gdal/bin/gdal-config")

yields an error with the following trace :

* installing *source* package ‘terra’ ...
** package ‘terra’ successfully unpacked and MD5 sums checked
** using staged installation
configure: CC: clang
configure: CXX: clang++ -std=gnu++11
configure: gdal-config set to /opt/homebrew/opt/gdal/bin/gdal-config
checking gdal-config exists... yes
checking gdal-config executable... yes
checking gdal-config usability... yes
configure: GDAL: 3.3.3
checking GDAL version >= 2.0.1... yes
checking for gcc... clang
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether clang accepts -g... yes
checking for clang option to accept ISO C89... none needed
checking how to run the C preprocessor... clang -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking gdal.h usability... yes
checking gdal.h presence... yes
checking for gdal.h... yes
checking GDAL: linking with --libs only... yes
checking GDAL: /opt/homebrew/Cellar/gdal/3.3.3_1/share/gdal/pcs.csv readable... no
checking GDAL: checking whether PROJ is available for linking:... yes
checking GDAL: checking whether PROJ is available fur running:... yes
configure: GDAL: 3.3.3
./configure: line 3657: pkg-config: command not found
checking proj.h usability... yes
checking proj.h presence... yes
checking for proj.h... yes
checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no
configure: error: libproj or sqlite3 not found in standard or given locations.
ERROR: configuration failed for package ‘terra’

[rhijmans]

Does this also fail with the development version?

[aourednik]

@rhijmans yes compilation fails with same trace when running remotes::install_github("rspatial/terra",configure.args = "--with-gdal-config=/opt/homebrew/opt/gdal/bin/gdal-config")

─ Session info ─────────────────────────────────────────────────────────────────────────────────────────────
 setting  value
 version  R version 4.1.2 (2021-11-01)
 os       macOS Monterey 12.0.1
 system   aarch64, darwin20.6.0
 ui       RStudio
 language (EN)
 collate  en_US.UTF-8
 ctype    en_US.UTF-8
 date     2021-12-06
 rstudio  2021.09.1+372 Ghost Orchid (desktop)
 pandoc   NA

─ Packages ─────────────────────────────────────────────────────────────────────────────────────────────────
 package     * version date (UTC) lib source
 callr         3.7.0   2021-04-20 [1] CRAN (R 4.1.2)
 cli           3.1.0   2021-10-27 [1] CRAN (R 4.1.2)
 crayon        1.4.2   2021-10-29 [1] CRAN (R 4.1.2)
 curl          4.3.2   2021-06-23 [1] CRAN (R 4.1.2)
 pkgbuild      1.2.1   2021-11-30 [1] CRAN (R 4.1.2)
 prettyunits   1.1.1   2020-01-24 [1] CRAN (R 4.1.2)
 processx      3.5.2   2021-04-30 [1] CRAN (R 4.1.2)
 ps            1.6.0   2021-02-28 [1] CRAN (R 4.1.2)
 R6            2.5.1   2021-08-19 [1] CRAN (R 4.1.2)
 remotes       2.4.2   2021-11-30 [1] CRAN (R 4.1.2)
 rprojroot     2.0.2   2020-11-15 [1] CRAN (R 4.1.2)
 rstudioapi    0.13    2020-11-12 [1] CRAN (R 4.1.2)
 sessioninfo * 1.2.2   2021-12-06 [1] CRAN (R 4.1.2)
 withr         2.4.3   2021-11-30 [1] CRAN (R 4.1.2)

 [1] /opt/homebrew/lib/R/4.1/site-library
 [2] /opt/homebrew/Cellar/r/4.1.2/lib/R/library

[rhijmans]

I looked at the issue you linked to, but it seems that the problem just magically disappeared there? Or was it the pull request you refer to, but that was applied to both terra and sf. So I am a bit at loss, not having access to a new mac myself. Can you show what is printed when you install sf?

[ax-sc]

I do have the same problem as @aourednik when trying to install terra (also when installing from GitHub).

pkg-config,gdal and proj are installed using Homebrew:

$ pkg-config --version
0.29.2

$ gdalinfo --version  
GDAL 3.3.3, released 2021/10/25

$ proj                
Rel. 8.2.0, November 1st, 2021

@rhijmans Let me know if I can provide any additional information. Here is the output when trying to install sf:

* installing *source* package ‘classInt’ ...
** package ‘classInt’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
gfortran -fno-optimize-sibling-calls  -fPIC  -g -O2  -c fish1.f -o fish1.o
make: gfortran: No such file or directory
make: *** [fish1.o] Error 1
ERROR: compilation failed for package ‘classInt’
* removing ‘/opt/homebrew/lib/R/4.1/site-library/classInt’
Warning in install.packages :
  installation of package ‘classInt’ had non-zero exit status
* installing *source* package ‘s2’ ...
** package ‘s2’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
** R
** data
*** moving datasets to lazyload DB
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (s2)
* installing *source* package ‘units’ ...
** package ‘units’ successfully unpacked and MD5 sums checked
** using staged installation
configure: units: 0.7-2
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether clang++ -std=gnu++14 accepts -g... yes
checking how to run the C++ preprocessor... clang++ -std=gnu++14 -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... rm: conftest.dSYM: is a directory
rm: conftest.dSYM: is a directory
yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for stdbool.h that conforms to C99... no
checking for _Bool... no
checking for error_at_line... no
checking for gcc... clang
checking whether we are using the GNU C compiler... yes
checking whether clang accepts -g... yes
checking for clang option to accept ISO C89... none needed
checking for XML_ParserCreate in -lexpat... yes
checking udunits2.h usability... no
checking udunits2.h presence... no
checking for udunits2.h... no
checking udunits2/udunits2.h usability... no
checking udunits2/udunits2.h presence... no
checking for udunits2/udunits2.h... no
checking for ut_read_xml in -ludunits2... no
configure: error: in `/private/var/folders/8d/s45_v8yd1d7f4lbbmwpjwzn80000gn/T/RtmpxWjRh1/R.INSTALL2fbb370ad0f1/units':
configure: error: 
--------------------------------------------------------------------------------
  Configuration failed because libudunits2.so was not found. Try installing:
    * deb: libudunits2-dev (Debian, Ubuntu, ...)
    * rpm: udunits2-devel (Fedora, EPEL, ...)
    * brew: udunits (OSX)
  If udunits2 is already installed in a non-standard location, use:
    --configure-args='--with-udunits2-lib=/usr/local/lib'
  if the library was not found, and/or:
    --configure-args='--with-udunits2-include=/usr/include/udunits2'
  if the header was not found, replacing paths with appropriate values.
  You can alternatively set UDUNITS2_INCLUDE and UDUNITS2_LIBS manually.
--------------------------------------------------------------------------------

See `config.log' for more details
ERROR: configuration failed for package ‘units’
* removing ‘/opt/homebrew/lib/R/4.1/site-library/units’
Warning in install.packages :
  installation of package ‘units’ had non-zero exit status
ERROR: dependencies ‘classInt’, ‘units’ are not available for package ‘sf’
* removing ‘/opt/homebrew/lib/R/4.1/site-library/sf’
Warning in install.packages :
  installation of package ‘sf’ had non-zero exit status

[kadyb]

@ax-sc, have you tried following the hints from the log?

  Configuration failed because libudunits2.so was not found. Try installing:
    * deb: libudunits2-dev (Debian, Ubuntu, ...)
    * rpm: udunits2-devel (Fedora, EPEL, ...)
    * brew: udunits (OSX)
  If udunits2 is already installed in a non-standard location, use:
    --configure-args='--with-udunits2-lib=/usr/local/lib'
  if the library was not found, and/or:
    --configure-args='--with-udunits2-include=/usr/include/udunits2'
  if the header was not found, replacing paths with appropriate values.
  You can alternatively set UDUNITS2_INCLUDE and UDUNITS2_LIBS manually.

Shouldn't you install gfortran too (if it is not installed)?

gfortran -fno-optimize-sibling-calls  -fPIC  -g -O2  -c fish1.f -o fish1.o
make: gfortran: No such file or directory
make: *** [fish1.o] Error 1
ERROR: compilation failed for package ‘classInt’

[ax-sc]

Thanks for your hints, @kadyb . gfortran is already installed (GNU Fortran (Homebrew GCC 11.2.0_3) 11.2.0). As i originally tried to install terra and not sf, i did not try to fix udunits before but rather provided the logs as they were to help troubleshooting this issue.

However i installed it now (using Homebrew) and it fixed this part of the error. Installing sf still fails due to gfortran being not found but i don´t know if this is the right place to try to fix this as it seems to me that it is not linked to the original issue?

[ax-sc]

I was able to compile terra as well as sf by using a R session in zsh without any additional changes (or arguments to the install command). So i think the original issue is more likely a problem with RStudio (I used the console in RStudio before) not finding pkg-config,gfortran, etc. and not with your package(s).

[rhijmans]

Thanks for letting us know. Someone else emailed me with a similar statement:

it seems that Rstudio grabs the path from some location that differs from the system $PATH variable. But when running R in a terminal terra installed fine from github.

[aourednik]

@ax-sc :

I confirm: install.packages("terra") from the MacOS terminal running R works fine !

For your Fortran not found problem, @ax-sc, I had that one too, despite gfortran being installed. Solved by adding these lines to ~/R./Makevars:

FC      = /opt/homebrew/bin/gfortran
F77     = /opt/homebrew/bin/gfortran

This did solve part of the sf issue but not the terra issue. Your workaround (using the MacOS terminal instead of RStudio) solved terra installation

[rhijmans]

I would think that if you can install sf, you should be able to also install terra, as they have the same install scripts (I copied them again today), and terra has one dependency less (udunits). It is a mystery to me why that would not be the case.

[ax-sc]

Thanks for letting us know. Someone else emailed me with a similar statement:

it seems that Rstudio grabs the path from some location that differs from the system $PATH variable. But when running R in a terminal terra installed fine from github.

It seems like this has been discussed before (https://github.com/rstudio/rstudio/issues/6482) and it is intentional to not pick up environment variables defined in .zshenv (or similiar) which makes the PATH differ and not include for example components installed with homebrew (as the homebrew dir has to be added to the PATH in .zshenv).

[samueldnj]

Hi all, I hate to bring this up again, but I'm getting the exact same error above (sqlite3 and PROJ not in expected location) when installing terra and sp from an R session on the terminal.

I've followed most of the tips above, and added the path to the binaries for the sqlite3 homebrew directory to my $PATH variable. Any insights?

I'm running MacOS Monterey 12.2.1, M1 Max.

> sessionInfo()
R version 4.1.2 (2021-11-01)
Platform: aarch64-apple-darwin21.1.0 (64-bit)
Running under: macOS Monterey 12.2.1

Matrix products: default
BLAS:   /opt/homebrew/Cellar/openblas/0.3.20/lib/libopenblasp-r0.3.20.dylib
LAPACK: /opt/homebrew/Cellar/r/4.1.2/lib/R/lib/libRlapack.dylib

locale:
[1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.1.2 tools_4.1.2   

Just to match the reported packages above

% pkg-config --version
0.29.2

% gdalinfo --version
GDAL 3.4.1, released 2021/12/27

% proj
Rel. 8.2.1, January 1st, 2022

% sqlite3 --version
3.38.0 2022-02-22 18:58:40 40fa792d359f84c3b9e9d6623743e1a59826274e221df1bde8f47086968a1bab

Output when installing terra using install.packages("terra",configure.args = "--with-gdal-config=/opt/homebrew/opt/gdal/bin/gdal-config").

* installing *source* package ‘terra’ ...
** package ‘terra’ successfully unpacked and MD5 sums checked
** using staged installation
configure: WARNING: unrecognized options: --with-sqlite-config
configure: CC: clang
configure: CXX: clang++ -std=gnu++11
configure: gdal-config set to /opt/homebrew/opt/gdal/bin/gdal-config
checking gdal-config exists... yes
checking gdal-config executable... yes
checking gdal-config usability... yes
configure: GDAL: 3.4.1
checking GDAL version >= 2.0.1... yes
checking for gcc... clang
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether clang accepts -g... yes
checking for clang option to accept ISO C89... none needed
checking how to run the C preprocessor... clang -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking gdal.h usability... yes
checking gdal.h presence... yes
checking for gdal.h... yes
checking GDAL: linking with --libs only... yes
checking GDAL: /opt/homebrew/Cellar/gdal/3.4.1_2/share/gdal/pcs.csv readable... no
checking GDAL: checking whether PROJ is available for linking:... yes
checking GDAL: checking whether PROJ is available fur running:... yes
configure: GDAL: 3.4.1
configure: pkg-config proj exists, will use it
configure: using proj.h.
configure: PROJ: 8.2.1
checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no
configure: error: libproj or sqlite3 not found in standard or given locations.
ERROR: configuration failed for package ‘terra’
* removing ‘/opt/homebrew/lib/R/4.1/site-library/terra’

Is there some sqlite3 config flag I can use? I tried --with-sqlite-config=/opt/homebrew/opt/sqlite/lib/pkgconfig but it didn't work.

[aourednik]

@samueldnj

It is by far easier to install the precompiled packages with this one-liner:

install.packages("https://cran.r-project.org/bin/macosx/big-sur-arm64/contrib/4.1/terra_1.5-21.tgz", repos = NULL, type = .Platform$pkgType)

Works even with a Homebrew installation of R. Unless you are a developper of the terra package, better to use the precompiled version.

[samueldnj]

Thanks Andre! That worked a treat. I will investigate this option for the other packages as well.

Sam

On Mar 2, 2022, at 3:56 PM, André Ourednik @.***> wrote:

@samueldnj https://github.com/samueldnj It is by far easier to install the precompiled packages with this one-liner:

install.packages("https://cran.r-project.org/bin/macosx/big-sur-arm64/contrib/4.1/terra_1.5-21.tgz", repos = NULL, type = .Platform$pkgType) Works even with a Homebrew installation of R. Unless you are a developper of the terra package, better to use the precompiled version.

— Reply to this email directly, view it on GitHub https://github.com/rspatial/terra/issues/429#issuecomment-1057518833, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQ2ZR2GZZ34WI73IBYHXNLU5753HANCNFSM5JNNDAAA. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you were mentioned.

[hzambran]

@samueldnj

It is by far easier to install the precompiled packages with this one-liner:

install.packages("https://cran.r-project.org/bin/macosx/big-sur-arm64/contrib/4.1/terra_1.5-21.tgz", repos = NULL, type = .Platform$pkgType)

Works even with a Homebrew installation of R. Unless you are a developper of the terra package, better to use the precompiled version.

Since yesterday Aug 07th, 2022, I have been trying to install terra on Mac OSX ARM, without succeeding.

Based on the recommendation of @aourednik, I tried the following:

install.packages("https://cran.r-project.org/bin/macosx/big-sur-arm64/contrib/4.2/terra_1.6-3.tgz", repos = NULL, type = .Platform$pkgType)

but I got the following error message:

trying URL 'https://cran.r-project.org/bin/macosx/big-sur-arm64/contrib/4.2/terra_1.6-3.tgz'
Content type 'application/x-gzip' length 104095350 bytes (99.3 MB)

downloaded 1.7 MB

Error in download.file(p, destfile, method, mode = "wb", ...) : 
  download from 'https://cran.r-project.org/bin/macosx/big-sur-arm64/contrib/4.2/terra_1.6-3.tgz' failed
In addition: Warning messages:
1: In download.file(p, destfile, method, mode = "wb", ...) :
  downloaded length 1736704 != reported length 104095350
2: In download.file(p, destfile, method, mode = "wb", ...) :
  URL 'https://cran.r-project.org/bin/macosx/big-sur-arm64/contrib/4.2/terra_1.6-3.tgz': Timeout of 60 seconds was reached

Also, I tried (a different mirror):

install.packages("terra", type="mac.binary")

but I got a similar error message:

There is a binary version available (and will be installed) but the
  source version is later:
      binary source
terra 1.5-21  1.6-7

trying URL 'https://cran.ma.imperial.ac.uk/bin/macosx/contrib/4.2/terra_1.5-21.tgz'
Content type 'application/x-gzip' length 94144562 bytes (89.8 MB)
=======
downloaded 13.4 MB

Error in download.file(url, destfile, method, mode = "wb", ...) : 
  download from 'https://cran.ma.imperial.ac.uk/bin/macosx/contrib/4.2/terra_1.5-21.tgz' failed
In addition: Warning messages:
1: In download.file(url, destfile, method, mode = "wb", ...) :
  downloaded length 14008320 != reported length 94144562
2: In download.file(url, destfile, method, mode = "wb", ...) :
  URL 'https://cran.ma.imperial.ac.uk/bin/macosx/contrib/4.2/terra_1.5-21.tgz': Timeout of 60 seconds was reached
Warning in download.packages(pkgs, destdir = tmpd, available = available,  :
  download of package ‘terra’ failed

I finally manage to get terra installed with this (https://github.com/rspatial/terra, https://github.com/r-spatial/sf/issues/1894):

install.packages("terra", type = "source", configure.args = c("--with-sqlite3-lib=/opt/homebrew/opt/sqlite/lib", "--with-proj-lib=/opt/homebrew/opt/proj/lib"))

However, after that installation, I got the following error in some situations:

Error in sp::CRS(...) : 
  PROJ4 argument-value pairs must begin with +: GEOGCRS["WGS 84",
  DATUM["World Geodetic System 1984",
 ELLIPSOID["WGS 84",6378137,298.257223563,
 LENGTHUNIT["metre",1]]],
 PRIMEM["Greenwich",0,
 ANGLEUNIT["degree",0.0174532925199433]],
 CS[ellipsoidal,2],
 AXIS["geodetic latitude (Lat)",north,
 ORDER[1],
 longitude (Lon)",east,
 ORDER[2],
 ID["EPSG",4326]]

which I solved with (https://github.com/rsbivand/whyR20_files/issues/1#issuecomment-702744904):

remotes::install_github("rsbivand/sp")
install.packages("rgdal", repos="http://R-Forge.R-project.org")

[swpark66]

I had the same problem. I solved it as follows.

install.packages('terra', configure.args=c('--with-gdal-config=/opt/homebrew/bin/gdal-config', '--with-pkg-config=/opt/homebrew/bin/pkg-config', '--with-sqlite3=/opt/homebrew/opt/sqlite3/bin/sqlite3', '--with-geos-config=/opt/homebrew/bin/geos-config', '--with-proj-include=/opt/homebrew/opt/proj@7/include/', '--with-proj-lib=/opt/homebrew/opt/proj@7/lib/', '--with-proj=/opt/homebrew/opt/proj@7'))

[pat-s]

Defining configure.args in install.packages() is not really a sustainable option imo, especially not for automated CI workflows and similar. The same goes for hardcoding a library to an old version (e.g. proj@7).

The workaround described in https://github.com/r-spatial/sf/issues/1894#issuecomment-1120383916 (and extended in https://github.com/r-spatial/sf/issues/1894#issuecomment-1211919141) is somewhat more generic and also applicable on CI systems during its generic definition which works for both {sf} and {terra} as both suffer from the same issue.