search

jeroen / openssl

OpenSSL bindings for R
Other
63 stars 19 forks source link

Mac OS Ventura R-devel: `symbol not found in flat namespace '_PKCS12_SAFEBAG_free'` #107

Closed MikeDMorgan closed 1 year ago

MikeDMorgan commented 1 year ago

I am attempting to install openssl for R-devel, which I need to make updates for my Bioconductor packages (lots of dependencies use openssl unsurprisingly). This issue appears related to #92 - the error is similar, i.e., symbol not found in flat namespace '_PKCS12_SAFEBAG_free'. However, I am using the openssl-1.1 M1 version linked in /opt/homebrew. I have tried the same with openssl v3 installation through homebrew which returns the same error.

The contents of my .R/Makevars:

% cat ~/.R/Makevars 
FLIBS   =-L/opt/homebrew/opt/gfortran/lib
F77     = /opt/homebrew/bin/gfortran
FC      = /opt/homebrew/bin/gfortran
CFLAGS   = -I/opt/homebrew/include
CPPFLAGS = -I/opt/homebrew/include
CXXFLAGS = -I/opt/homebrew/include

Error message: 

> install.packages("openssl", repos="https://cran.ma.imperial.ac.uk/")
Warning: unable to access index for repository https://cran.ma.imperial.ac.uk/bin/macosx/contrib/4.3:
  cannot open URL 'https://cran.ma.imperial.ac.uk/bin/macosx/contrib/4.3/PACKAGES'
Package which is only available in source form, and may need
  compilation of C/C++/Fortran: ‘openssl’
Do you want to attempt to install these from sources? (Yes/no/cancel) Yes
installing the source package ‘openssl’

trying URL 'https://cran.ma.imperial.ac.uk/src/contrib/openssl_2.0.5.tar.gz'
Content type 'application/x-gzip' length 1203925 bytes (1.1 MB)
==================================================
downloaded 1.1 MB

* installing *source* package ‘openssl’ ...
** package ‘openssl’ successfully unpacked and MD5 sums checked
** using staged installation
Found pkg-config cflags and libs!
Using PKG_CFLAGS=-I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include
Using PKG_LIBS=-L/opt/homebrew/Cellar/openssl@1.1/1.1.1t/lib -lssl -lcrypto
** libs
using C compiler: ‘Apple clang version 14.0.0 (clang-1400.0.29.202)’
using SDK: ‘’
rm -f aes.o base64.o bignum.o cert.o compatibility.o diffie.o envelope.o error.o hash.o info.o keygen.o keys.o onload.o openssh.o password.o pbkdf.o pem.o pkcs12.o pkcs7.o rand.o rsa.o signing.o ssl.o stream.o write.o x25519.o openssl.so bcrypt/libstatbcrypt.a bcrypt/bcrypt_pbkdf.o bcrypt/blowfish.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c aes.c -o aes.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c base64.c -o base64.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c bignum.c -o bignum.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c cert.c -o cert.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c compatibility.c -o compatibility.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c diffie.c -o diffie.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c envelope.c -o envelope.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c error.c -o error.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c hash.c -o hash.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c info.c -o info.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c keygen.c -o keygen.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c keys.c -o keys.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c onload.c -o onload.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c openssh.c -o openssh.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c password.c -o password.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c pbkdf.c -o pbkdf.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c pem.c -o pem.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c pkcs12.c -o pkcs12.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c pkcs7.c -o pkcs7.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c rand.c -o rand.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c rsa.c -o rsa.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c signing.c -o signing.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c ssl.c -o ssl.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c stream.c -o stream.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c write.c -o write.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c x25519.c -o x25519.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c bcrypt/bcrypt_pbkdf.c -o bcrypt/bcrypt_pbkdf.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/opt/homebrew/Cellar/openssl@1.1/1.1.1t/include  -I/opt/homebrew/include    -fPIC  -I/opt/homebrew/include -c bcrypt/blowfish.c -o bcrypt/blowfish.o
ar rcs bcrypt/libstatbcrypt.a bcrypt/bcrypt_pbkdf.o bcrypt/blowfish.o
clang -mmacosx-version-min=10.13 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o openssl.so aes.o base64.o bignum.o cert.o compatibility.o diffie.o envelope.o error.o hash.o info.o keygen.o keys.o onload.o openssh.o password.o pbkdf.o pem.o pkcs12.o pkcs7.o rand.o rsa.o signing.o ssl.o stream.o write.o x25519.o -Lbcrypt -lstatbcrypt -L/opt/homebrew/Cellar/openssl@1.1/1.1.1t/lib -lssl -lcrypto -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: ignoring file /opt/homebrew/Cellar/openssl@1.1/1.1.1t/lib/libcrypto.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /opt/homebrew/Cellar/openssl@1.1/1.1.1t/lib/libssl.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
installing to /Library/Frameworks/R.framework/Versions/R-devel/Versions/4.3/Resources/library/00LOCK-openssl/00new/openssl/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘openssl’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Library/Frameworks/R.framework/Versions/R-devel/Versions/4.3/Resources/library/00LOCK-openssl/00new/openssl/libs/openssl.so':
  dlopen(/Library/Frameworks/R.framework/Versions/R-devel/Versions/4.3/Resources/library/00LOCK-openssl/00new/openssl/libs/openssl.so, 0x0006): symbol not found in flat namespace '_PKCS12_SAFEBAG_free'
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Library/Frameworks/R.framework/Versions/R-devel/Versions/4.3/Resources/library/openssl’

The downloaded source packages are in
    ‘/private/var/folders/p7/jt1p0wfj39xc_mr4026j8zvhry786r/T/RtmpuA1Hn6/downloaded_packages’
Warning message:
In install.packages("openssl", repos = "https://cran.ma.imperial.ac.uk/") :
  installation of package ‘openssl’ had non-zero exit status
> sessionInfo()
R Under development (unstable) (2023-01-18 r83642)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Big Sur ... 10.16

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/R-devel/Versions/4.3/Resources/lib/libRblas.0.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/R-devel/Versions/4.3/Resources/lib/libRlapack.dylib;  LAPACK version 3.11.0

locale:
[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8

time zone: Europe/London
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.3.0 tools_4.3.0

Thanks in advance for your help.

jeroen commented 1 year ago

The problem is shown in this line of your output

ld: warning: ignoring file /opt/homebrew/Cellar/openssl@1.1/1.1.1t/lib/libcrypto.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /opt/homebrew/Cellar/openssl@1.1/1.1.1t/lib/libssl.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64

Which build of R are you using? It seems to be an intel build running under the rosetta emulator. Please try installing the arm64 version of R from: https://cran.r-project.org/bin/macosx/

MikeDMorgan commented 1 year ago

Hi @jeroen - as I stated, I am working on R-devel installed using the arm64 package builder: https://mac.r-project.org/. OpenSSL is installed via homebrew, which only has builds for x86_64. When using the in-built LibreSSL v3.3.6 or Macports openssl-3 install.packages("openssl") complains that openssl isn't installed:

> install.packages("openssl", repos="https://cran.ma.imperial.ac.uk/")
Warning: unable to access index for repository https://cran.ma.imperial.ac.uk/bin/macosx/contrib/4.3:
  cannot open URL 'https://cran.ma.imperial.ac.uk/bin/macosx/contrib/4.3/PACKAGES'
Package which is only available in source form, and may need
  compilation of C/C++/Fortran: ‘openssl’
Do you want to attempt to install these from sources? (Yes/no/cancel) Yes
installing the source package ‘openssl’

trying URL 'https://cran.ma.imperial.ac.uk/src/contrib/openssl_2.0.5.tar.gz'
Content type 'application/x-gzip' length 1203925 bytes (1.1 MB)
==================================================
downloaded 1.1 MB

* installing *source* package ‘openssl’ ...
** package ‘openssl’ successfully unpacked and MD5 sums checked
** using staged installation
Homebrew 4.0.1
Homebrew/homebrew-core (git revision 555e4e0e281; last commit 2023-02-20)
Homebrew/homebrew-cask (git revision 722b9ee1a7; last commit 2023-02-20)
Using PKG_CFLAGS=-I/opt/homebrew/opt/openssl/include -I/opt/homebrew/opt/openssl@1.1/include
--------------------------- [ANTICONF] --------------------------------
Configuration failed because openssl was not found. Try installing:
 * deb: libssl-dev (Debian, Ubuntu, etc)
 * rpm: openssl-devel (Fedora, CentOS, RHEL)
 * csw: libssl_dev (Solaris)
 * brew: openssl (Mac OSX)
If openssl is already installed, check that 'pkg-config' is in your
PATH and PKG_CONFIG_PATH contains a openssl.pc file. If pkg-config
is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:
R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'
-------------------------- [ERROR MESSAGE] ---------------------------
tools/version.c:1:10: fatal error: 'openssl/opensslv.h' file not found
#include <openssl/opensslv.h>
         ^~~~~~~~~~~~~~~~~~~~
1 error generated.
--------------------------------------------------------------------
ERROR: configuration failed for package ‘openssl’
* removing ‘/Library/Frameworks/R.framework/Versions/R-devel/Versions/4.3/Resources/library/openssl’

The downloaded source packages are in
    ‘/private/var/folders/p7/jt1p0wfj39xc_mr4026j8zvhry786r/T/RtmplkXoBU/downloaded_packages’
Warning message:
In install.packages("openssl", repos = "https://cran.ma.imperial.ac.uk/") :
  installation of package ‘openssl’ had non-zero exit status

R and/or install.packages is ignoring my user-set $PKG_CONFIG_PATH which is set to the macports version:

% echo $PKG_CONFIG_PATH
/opt/local/libexec/openssl-3/lib/pkgconfig

LDFLAGS and CPPFLAGS are also set accorindingly:

export LDFLAGS="-L/opt/local/libexec/openssl-3/lib"
export CPPFLAGS="-I/opt/local/libexec/openssl-3/include"

as are the C**FLAGS in .R/Makevars:

CFLAGS   = -I/opt/local/include
CPPFLAGS = -I/opt/local/include
CXXFLAGS = -I/opt/local/include
jeroen commented 1 year ago

The version of R you are using above is certainly not the arm64 package, because you are showing us lines with:

-mmacosx-version-min=10.13

Which is only used for intel builds. Please try reinstalling this one: https://mac.r-project.org/big-sur/R-devel/R-devel.pkg

OpenSSL is installed via homebrew, which only has builds for x86_64.

That is not true, homebrew has arm64 builds. According to your own output here, the problem is that R is an intel build and homebrew is using arm:

ld: warning:..... building for macOS-x86_64 but attempting to link with file built for macOS-arm64

Something else you can try: if there is no openssl found locally, and you set a environment variable CI=1 it will automatically download a suitable version of libssl. Like so:

Sys.setenv(CI=1)
install.packages("openssl")
MikeDMorgan commented 1 year ago

Thanks @jeroen - I think the R-devel package I downloaded last week was linked to the wrong version then. Todays R-devel build works, sorry for the hassle.