Support binary installation from r-universe on arm64 Mac

[rossellhayes]

r-universe now builds arm64 binaries for macOS. However, when installing a package from an r-universe repo with arm64 builds available, pak still attempts to install from source.

pak::pkg_install("arrow")
#> ✔ Loading metadata database ... done
#>                                                                             
#> → Will install 1 package.
#> → The package (0 B) is cached.
#> + arrow   14.0.2.9000 👷🏽‍♂️🔧
#> ℹ No downloads are needed, 1 pkg is cached
#> ✔ Got arrow 14.0.2.9000 (source) (4.76 MB)
#> ℹ Building arrow 14.0.2.9000

In comparison, install.packages() can install the binary if the type argument is set appropriately.

install.packages("arrow", type = "mac.binary")
#> Installing package into ‘/opt/homebrew/lib/R/4.3/site-library’
#> (as ‘lib’ is unspecified)
#> trying URL 'https://apache.r-universe.dev/bin/macosx/contrib/4.3/arrow_14.0.2.9000.tgz'
#> Content type 'application/x-gzip' length 18204809 bytes (17.4 MB)
#> ==================================================
#> downloaded 17.4 MB
#> 
#> 
#> The downloaded binary packages are in
#>   /var/folders/v_/rg1xs3fx2tj1n643_xh87r6c0000gq/T//RtmpQEpbSB/downloaded_packages

[gaborcsardi]

Did you actually try to use that package that you installed with install.packages()? Because I am fairly sure that that's an x86_64 package. You can get an arm64 package with type = "mac.binary.big-sur-arm64":

> install.packages("arrow", repos = "https://apache.r-universe.dev", type = "mac.binary.big-sur-arm64")
Installing package into ‘/opt/homebrew/lib/R/4.3/site-library’
(as ‘lib’ is unspecified)
Warning: dependencies ‘assertthat’, ‘bit64’, ‘glue’, ‘purrr’, ‘R6’, ‘rlang’, ‘tidyselect’, ‘vctrs’, ‘cpp11’ are not available
trying URL 'https://apache.r-universe.dev/bin/macosx/big-sur-arm64/contrib/4.3/arrow_14.0.2.9000.tgz'
Content type 'application/x-gzip' length 15475459 bytes (14.8 MB)
==================================================
downloaded 14.8 MB

But of course now the dependencies are not installed. If you add a CRAN repo as well, then the dependencies are also installed, but from CRAN binaries, which are not compatible with homebrew R:

> install.packages("arrow", repos = c("https://apache.r-universe.dev", "https://cloud.r-project.org"), type = "mac.binary.big-sur-arm64")
[...]
> library(arrow)

 *** caught segfault ***
address 0x0, cause 'invalid permissions'

Traceback:
 1: dyn.load(file, DLLpath = DLLpath, ...)
 2: library.dynam(lib, package, package.lib)
 3: loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]])

So this is not so simple, I am afraid.

We could potentially make pak smart about this, so it installs binaries from R-universe on homebrew R, but sourced from CRAN.

But do R-universe arm64 macos binaries work on homebrew R at all?

[gaborcsardi]

Yeah, so it seems that R-universe binaries (like CRAN binaries) do not work on homebrew R.

> install.packages("ps", repos = "https://r-lib.r-universe.dev/", type = "mac.binary.big-sur-arm64")
Installing package into '/opt/homebrew/lib/R/4.3/site-library'
(as 'lib' is unspecified)
trying URL 'https://r-lib.r-universe.dev/bin/macosx/big-sur-arm64/contrib/4.3/ps_1.7.5.9000.tgz'
Content type 'application/x-gzip' length 314955 bytes (307 KB)
==================================================
downloaded 307 KB

The downloaded binary packages are in
    /var/folders/ph/fpcmzfd16rgbbk8mxvy9m2_h0000gn/T//RtmpmRfOaP/downloaded_packages
> library(ps)

 *** caught segfault ***
address 0x18, cause 'invalid permissions'

Traceback:
 1: dyn.load(file, DLLpath = DLLpath, ...)

On CRAN's R, pak uses the binaries happily by default:

❯ pak::repo_add(APACHE = "https://apache.r-universe.dev")
❯ pak::meta_list(pkg="arrow")[, c("package", "version", "type", "platform")]
# A data frame: 4 × 4
  package version     type     platform
* <chr>   <chr>       <chr>    <chr>
1 arrow   14.0.0.2    cran     aarch64-apple-darwin20
2 arrow   14.0.0.2    cran     source
3 arrow   14.0.2.9000 cranlike aarch64-apple-darwin20
4 arrow   14.0.2.9000 cranlike source

❯ pak::pkg_install("arrow?nocache&reinstall")

→ Will install 1 package.
→ Will download 1 package with unknown size.
+ arrow   14.0.2.9000 🔧 ⬇
ℹ Getting 1 pkg with unknown size
✔ Got arrow 14.0.2.9000 (aarch64-apple-darwin20) (15.48 MB)
✔ Downloaded 1 package (15.48 MB) in 1.7s
✔ Installed arrow 14.0.2.9000  (86ms)
✔ 1 pkg + 13 deps: kept 11, added 1, dld 1 (15.48 MB) [2.1s]

[gaborcsardi]

We tried to fix R-universe binaries so they work on homebrew R, but eventually we could not solve all issues. Rewriting the full link path to the library name works for some packages, but others still fail to load, e.g. nlme:

❯ R_JAVA_LD_LIBRARY_PATH=/opt/homebrew/opt/gcc/lib/gcc/current /opt/homebrew/Cellar/r/4.3.2/bin/R -q --vanilla -e 'library(nlme)'
> library(nlme)
Error: package or namespace load failed for ‘nlme’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Users/gaborcsardi/Library/R/arm64/4.3/library/nlme/libs/nlme.so':
  dlopen(/Users/gaborcsardi/Library/R/arm64/4.3/library/nlme/libs/nlme.so, 0x0006): Symbol not found: _libintl_dgettext
  Referenced from: <4287F84C-0536-3FD2-9682-15EF0EBFA72A> /Users/gaborcsardi/Library/R/arm64/4.3/library/nlme/libs/nlme.so
  Expected in:     <58793B9F-D6CA-39F6-847F-02E8E211CA34> /opt/homebrew/Cellar/r/4.3.2/lib/R/lib/libR.dylib
Execution halted

(R_JAVA_LD_LIBRARY_PATH is a trick to add a path to the library path.)

Homebrew R links to /opt/homebrew/opt/gettext/lib/libintl.8.dylib, which has _libintl_dgettext, whereas CRAN R does not link to libintl dynamically, so it probably links to it statically.

[rossellhayes]

Thanks for the explanation @gaborcsardi, I didn't realize that installing from homebrew caused issues. I've reinstalled with the CRAN release and things are going smoothly!