`mvnfast` package looks for libgfortran.5.dylib in the wrong place

[Aariq]

Apologies if this belongs in the issues for mvnfast, but I thought I'd start here because I'm only having an issue when using renv.

I'm able to install mvnfast from CRAN both with and without renv active, but when I load mvnfast in the renv project, I get this:

Error: package or namespace load failed for ‘mvnfast’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Users/ericscott/Library/Caches/org.R-project.R/R/renv/cache/v5/R-4.2/x86_64-apple-darwin17.0/mvnfast/0.2.8/e65cac8e8501bdfbdca0412c37bb18c9/mvnfast/libs/mvnfast.so':
  dlopen(/Users/ericscott/Library/Caches/org.R-project.R/R/renv/cache/v5/R-4.2/x86_64-apple-darwin17.0/mvnfast/0.2.8/e65cac8e8501bdfbdca0412c37bb18c9/mvnfast/libs/mvnfast.so, 0x0006): Library not loaded: /osxcross/bin/../x86_64-apple-darwin22/lib/libgfortran.5.dylib
  Referenced from: <3427F2BC-BE3C-3DE8-B5EA-F88A7475BF9A> /Users/ericscott/Library/Caches/org.R-project.R/R/renv/cache/v5/R-4.2/x86_64-apple-darwin17.0/mvnfast/0.2.8/e65cac8e8501bdfbdca0412c37bb18c9/mvnfast/libs/mvnfast.so
  Reason: tried: '/osxcross/bin/../x86_64-apple-darwin22/lib/libgfortran.5.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/osxcross/bin/../x86_64-apple-darwin22/lib/libgfortran.5.dylib' (no such fil

The problem here is that I don't have a /System/Volumes/Preboot/Cryptexes/OS/osxcross/ directory. The output of list.files(R.home("lib"), full.names = TRUE) is the same both with and without renv active.

list.files(R.home("lib"), full.names = TRUE)
 [1] "/Library/Frameworks/R.framework/Resources/lib/libgcc_s.1.dylib"          
 [2] "/Library/Frameworks/R.framework/Resources/lib/libgfortran.5.dylib"       
 [3] "/Library/Frameworks/R.framework/Resources/lib/libquadmath.0.dylib"       
 [4] "/Library/Frameworks/R.framework/Resources/lib/libR.dylib"                
 [5] "/Library/Frameworks/R.framework/Resources/lib/libR.dylib.dSYM"           
 [6] "/Library/Frameworks/R.framework/Resources/lib/libRblas.0.dylib"          
 [7] "/Library/Frameworks/R.framework/Resources/lib/libRblas.0.dylib.dSYM"     
 [8] "/Library/Frameworks/R.framework/Resources/lib/libRblas.dylib"            
 [9] "/Library/Frameworks/R.framework/Resources/lib/libRblas.dylib.dSYM"       
[10] "/Library/Frameworks/R.framework/Resources/lib/libRblas.vecLib.dylib"     
[11] "/Library/Frameworks/R.framework/Resources/lib/libRblas.vecLib.dylib.dSYM"
[12] "/Library/Frameworks/R.framework/Resources/lib/libRlapack.dylib"          
[13] "/Library/Frameworks/R.framework/Resources/lib/libRlapack.dylib.dSYM"     

`sessionInfo()`

``` R version 4.2.2 (2022-10-31) Platform: x86_64-apple-darwin17.0 (64-bit) Running under: macOS Ventura 13.4.1 Matrix products: default LAPACK: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8 attached base packages: [1] stats graphics grDevices datasets utils methods base loaded via a namespace (and not attached): [1] Rcpp_1.0.11 pillar_1.9.0 compiler_4.2.2 later_1.3.1 [5] data.validator_0.2.0 tools_4.2.2 digest_0.6.33 tsibble_1.1.3 [9] lubridate_1.9.2 evaluate_0.21 lifecycle_1.0.3 tibble_3.2.1 [13] anytime_0.3.9 timechange_0.2.0 pkgconfig_2.0.3 rlang_1.1.1 [17] shiny_1.7.4.1 cli_3.6.1 rstudioapi_0.14 xfun_0.39 [21] warp_0.2.0 fastmap_1.1.1 xml2_1.3.5 dplyr_1.1.2 [25] knitr_1.43 generics_0.1.3 vctrs_0.6.3 tidyselect_1.2.0 [29] glue_1.6.2 R6_2.5.1 fansi_1.0.4 rmarkdown_2.23 [33] slider_0.3.0 magrittr_2.0.3 promises_1.2.0.1 htmltools_0.5.5 [37] ellipsis_0.3.2 gt_0.9.0 xtable_1.8-4 mime_0.12 [41] renv_1.0.2 httpuv_1.6.11 utf8_1.2.3 ```

[kevinushey]

It seems to work for me:

> renv::install("mvnfast")
# Downloading packages -------------------------------------------------------
- Downloading mvnfast from CRAN ...             OK [437.2 Kb in 1.0s]
- Downloading RcppArmadillo from CRAN ...       OK [1.3 Mb in 0.78s]
Successfully downloaded 2 packages in 2.4 seconds.

The following package(s) will be installed:
- BH            [1.81.0-1]
- mvnfast       [0.2.8]
- Rcpp          [1.0.11]
- RcppArmadillo [0.12.6.1.0]
These packages will be installed into "/private/var/folders/9n/3nxsgkrj0rlfx196kzdttpww0000gn/T/tmp.MqPr8oeh/renv/library/R-4.3/aarch64-apple-darwin20".

Do you want to proceed? [Y/n]:
# Installing packages --------------------------------------------------------
- Installing Rcpp ...                           OK [linked from cache]
- Installing RcppArmadillo ...                  OK [built from source and cached in 13s]
- Installing BH ...                             OK [linked from cache]
- Installing mvnfast ...                        OK [built from source and cached in 18s]
Successfully installed 4 packages in 30 seconds.

Can you provide any other details? What is the output of Sys.getenv("DYLD_FALLBACK_LIBRARY_PATH")? The output of renv::diagnostics() might also be useful.

[jabenninghoff]

I've run into a similar problem with packages from https://packagemanager.posit.co (PPM), and can reproduce the issue with the following:

> renv::install("mvnfast", repos = "https://packagemanager.posit.co/cran/latest")
# Downloading packages -------------------------------------------------------
- Downloading mvnfast from CRAN ...             OK [558.7 Kb in 0.72s]
- Downloading BH from CRAN ...                  OK [12.3 Mb in 1.3s]
Successfully downloaded 2 packages in 4.2 seconds.

The following package(s) will be installed:
- BH      [1.81.0-1]
- mvnfast [0.2.8]
These packages will be installed into "~/Library/Caches/org.R-project.R/R/renv/library/rtraining-be68765c/R-4.3/x86_64-apple-darwin20".

Do you want to proceed? [Y/n]: y

# Installing packages --------------------------------------------------------
- Installing BH ...                             OK [installed binary and cached in 20s]
- Installing mvnfast ...                        OK [installed binary and cached in 0.7s]
Successfully installed 2 packages in 21 seconds.
> library(mvnfast)
Error: package or namespace load failed for ‘mvnfast’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Users/jabenninghoff/Library/Caches/org.R-project.R/R/renv/cache/v5/R-4.3/x86_64-apple-darwin20/mvnfast/0.2.8/e65cac8e8501bdfbdca0412c37bb18c9/mvnfast/libs/mvnfast.so':
  dlopen(/Users/jabenninghoff/Library/Caches/org.R-project.R/R/renv/cache/v5/R-4.3/x86_64-apple-darwin20/mvnfast/0.2.8/e65cac8e8501bdfbdca0412c37bb18c9/mvnfast/libs/mvnfast.so, 0x0006): Library not loaded: '/osxcross/bin/../x86_64-apple-darwin22/lib/libgfortran.5.dylib'
  Referenced from: '/Users/jabenninghoff/Library/Caches/org.R-project.R/R/renv/cache/v5/R-4.3/x86_64-apple-darwin20/mvnfast/0.2.8/e65cac8e8501bdfbdca0412c37bb18c9/mvnfast/libs/mvnfast.so'
  Reason: tried: '/osxcross/bin/../x86_64-apple-darwin22/lib/libgfortran.5.dylib' (no such file), '/usr/local/lib/libgfortran.5.dylib' (no such file), '/usr/lib/libgfortran.5.dylib' (no such file)

Reinstalling the package from CRAN fixes the issue:

> renv::install("mvnfast", repos = "https://cloud.r-project.org", rebuild = TRUE)
# Downloading packages -------------------------------------------------------
- Downloading mvnfast from CRAN ...             OK [file is up to date]
Successfully downloaded 1 package in 0.45 seconds.

The following package(s) will be installed:
- mvnfast [0.2.8]
These packages will be installed into "~/Library/Caches/org.R-project.R/R/renv/library/rtraining-be68765c/R-4.3/x86_64-apple-darwin20".

# Installing packages --------------------------------------------------------
- Installing mvnfast ...                        OK [installed binary and cached in 0.67s]
Successfully installed 1 package in 0.74 seconds.
> library(mvnfast)

I've had other problems as well - rendering an .Rmd file broke with packages from PPM, and now only use CRAN - from my renv.lock:

  "R": {
    "Version": "4.3.1",
    "Repositories": [
      {
        "Name": "CRAN",
        "URL": "https://cloud.r-project.org"
      }
    ]
  },

[fh-mthomson]

Starting ~08/03, we've seen ~5 users face this issue where libgfortran.5.dylib cannot be found when loading via (library(X)), across igraph, MatchIt, etc. We recently switched from external CRAN (https://cran.r-project.org/) to PPM.

Additional context: running on Mac (both M1 and Intel chips), R version 4.1.3

[jabenninghoff]

Looking at it again, my error isn't quite the same, but it seems like the common issue is that neither @Aariq's nor my environment conforms to what PPM uses to build packages... and I couldn't find any documentation on the PPM site that was helpful, nor anything similar on https://support.posit.co.

[kevinushey]

What is the output of Sys.getenv("DYLD_FALLBACK_LIBRARY_PATH")?

This looks like an issue with the binaries being produced by Posit Package Manager -- for mvtnorm, I see:

kevin@MBP-P2MQ:~/Library/R/x86_64/4.2/library/mvtnorm/libs
$ otool -L mvtnorm.so
mvtnorm.so:
        mvtnorm.so (compatibility version 0.0.0, current version 0.0.0)
        /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib (compatibility version 4.2.0, current version 4.2.0)
        /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRblas.dylib (compatibility version 0.0.0, current version 0.0.0)
        /osxcross/bin/../x86_64-apple-darwin22/lib/libgfortran.5.dylib (compatibility version 6.0.0, current version 6.0.0)
        /osxcross/bin/../x86_64-apple-darwin22/lib/libquadmath.0.dylib (compatibility version 1.0.0, current version 1.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
        /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1953.1.0)

And those library paths for libgfortran and libquadmath are incorrect; they should be linking to the R versions. However, normally those would be found via the DYLD_FALLBACK_LIBRARY_PATH, so I wonder why that's not working. It might be working for me just because I have SIP disabled:

$ csrutil status
System Integrity Protection status: disabled.

[fh-mthomson]

> Sys.getenv("DYLD_FALLBACK_LIBRARY_PATH")
[1] ""

csrutil status
System Integrity Protection status: enabled.

[kevinushey]

The fact that DYLD_FALLBACK_LIBRARY_PATH is empty is unexpected. Is there anything unique about how you're using R? If you're using RStudio, what version are you using?

[fh-mthomson]

I'm using vanilla RStudio, 2023.6.1.524 (same result either inside or outside of an renv project).

> library(MatchIt)
Error: package or namespace load failed for ‘MatchIt’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Library/Frameworks/R.framework/Versions/4.1/Resources/library/MatchIt/libs/MatchIt.so':
  dlopen(/Library/Frameworks/R.framework/Versions/4.1/Resources/library/MatchIt/libs/MatchIt.so, 0x0006): Library not loaded: '/osxcross/bin/../x86_64-apple-darwin22/lib/libgfortran.5.dylib'
  Referenced from: '/Library/Frameworks/R.framework/Versions/4.1/Resources/library/MatchIt/libs/MatchIt.so'
  Reason: tried: '/osxcross/bin/../x86_64-apple-darwin22/lib/libgfortran.5.dylib' (no such file), '/usr/local/lib/libgfortran.5.dylib' (no such file), '/usr/lib/libgfortran.5.dylib' (no such file)

However, if I run an R process through the terminal:

> Sys.getenv("DYLD_FALLBACK_LIBRARY_PATH")
[1] "/Library/Frameworks/R.framework/Resources/lib:/Library/Java/JavaVirtualMachines/jdk1.8.0_241.jdk/Contents/Home/jre/lib/server"

aha, then, success:

library(MatchIt)

[jabenninghoff]

DYLD_FALLBACK_LIBRARY_PATH is empty for me as well. I'm using R 4.3.1 installed by rig and using RStudio Version 2023.06.1+524 (2023.06.1+524). It doesn't appear to be set by default anywhere within macOS or in R.

> Sys.getenv("DYLD_FALLBACK_LIBRARY_PATH")
[1] ""

For comparison, here is what mvtnorm downloaded from CRAN shows for otool -L:

% otool -L mvtnorm.so
mvtnorm.so:
    mvtnorm.so (compatibility version 0.0.0, current version 0.0.0)
    /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/lib/libRlapack.dylib (compatibility version 4.3.0, current version 4.3.0)
    /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/lib/libRblas.dylib (compatibility version 0.0.0, current version 0.0.0)
    /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/lib/libgfortran.5.dylib (compatibility version 6.0.0, current version 6.0.0)
    /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/lib/libquadmath.0.dylib (compatibility version 1.0.0, current version 1.0.0)
    /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/lib/libR.dylib (compatibility version 4.3.0, current version 4.3.0)
    /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1775.118.101)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.100.5)

To me, this appears to be a problem with Posit Package Manager.

[jabenninghoff]

Interesting, I am also finding that DYLD_FALLBACK_LIBRARY_PATH is set in R terminal but not in RStudio:

R terminal:

> Sys.getenv("DYLD_FALLBACK_LIBRARY_PATH")
[1] "/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/lib:/Library/Java/JavaVirtualMachines/jdk-11.0.18+10/Contents/Home/lib/server"

[jabenninghoff]

This does seem to be related to https://github.com/rstudio/rstudio/issues/13085 : I have SIP enabled on my system.

[Aariq]

I also get an empty string for Sys.getenv("DYLD_FALLBACK_LIBRARY_PATH"). The only thing unusual about how I'm using R/RStudio is that I recently started using rig to manage R installations

[Aariq]

I was able to install the GitHub version of mvnfast and load it with no issue, but as far as I can tell from commit history it is not different from the version on CRAN.

[kevinushey]

Thanks everyone -- we've confirmed there is an issue on the Package Manager side; the team is now investigating. The incorrect linker paths, combined with the missing DYLD_FALLBACK_LIBRARY_PATH (as described in https://github.com/rstudio/rstudio/issues/13085), together culminate in the issues we're seeing here.

We'll also seek to resolve the DYLD_FALLBACK_LIBRARY_PATH issues for the next RStudio release.

[fh-mthomson]

@kevinushey thanks to you and the team for working in this!

In the meantime, do you have a recommendation between either (1) re-installing packages from CRAN instead of PPM (pending a release of PPM) or (2) setting DYLD_FALLBACK_LIBRARY_PATH manually (pending the release of Desert Sunflower IDE version)?

[kevinushey]

IMHO, the simplest fix is to just reinstall the affected packages from CRAN. However, if you're using the public instance of Posit Package Manager, those binaries should already be repaired -- so it should suffice to reinstall those packages from PPM as well.

I'm not sure if the fix has been published for on-prem installations of package manager, though.