Hi,

Context

This is part of an effort to create a fully reproducible software environment for computational pipelines, including versions of R packages from CRAN, Bioconductor, and GitHub. Using the Snakemake framework in a first instance, the objective is to use conda to manage the installation of R (base) and the renv package; while subsequently managing the installation of specific versions of R packages using renv (which uses a renv.lock file to track packages, versions, and source repositories).

Issue

When running renv::init() (in this case, equivalent to renv::restore() (see MWE further below), the installation of R packages crashes for Rhdf5lib due to compilation issues. This compilation issue may be related to the Conda environment and its local installation of R and dependencies.

The full error log is available at https://gist.github.com/kevinrue/85f7972763089a3cd15b4e27645c9000 To reproduce it, my current MWE is available with git clone -b renv git@github.com:kevinrue/snakemake_alevin_10x.git. The log above refers specifically to commit 7a95b7c.

This MWE still has a couple of pre-requisites:

install conda: miniconda
install snakemake

At this point you should have a Conda environment that includes snakemake, in this activated environment you can then run the following to reproduce the issue:

git clone -b renv https://github.com/kevinrue/snakemake_alevin_10x
cd snakemake_alevin_10x
git checkout 7a95b7c
snakemake --use-conda --cores 2 renv # set up the R environment

Related issues

Reading existing issues, I find common points with #21 and #31

Namely:

I install r-base (4.0.2) and r-renv (0.11.0) using Conda (like in https://github.com/grimbough/Rhdf5lib/issues/21#issue-465883700) In my MWE, see https://github.com/kevinrue/snakemake_alevin_10x/blob/renv/workflow/envs/r.yaml

Outside of Conda, I can install and load Rhdf5lib from source successfully (like in https://github.com/grimbough/Rhdf5lib/issues/21#issuecomment-513119881 point 2) i.e., the issue is specific to installing R 'within R' using BiocManager::install("Rhdf5lib") or equivalent at the R prompt within the Conda environment; whereas the Conda package bioconductor-rhdf5lib installs successfully.

My error happens at the exact same point as https://github.com/grimbough/Rhdf5lib/issues/31#issue-643391238 in the log file, but I do get a different error message, namely:

dyld: Library not loaded: @rpath/libcrypto.1.1.dylib
  Referenced from: /private/var/folders/51/nz__2dx10yzd75rylr5cg2zc0000gq/T/RtmpDvehtP/R.INSTALL13e51337443be/Rhdf5lib/src/hdf5/src/./H5make_libsettings
  Reason: image not found

Note also that the compilation process then carries on a bit more before failing with the final error:

cp: hdf5/src/.libs/libhdf5.a: No such file or directory
make: *** [Makevars:38: copying] Error 1
ERROR: compilation failed for package ‘Rhdf5lib’

Further notes

I have another MWE that does not run into the issue and completes successfully, as it does not install Rhdf5lib or any other package that requires a similar kind of compilation: https://github.com/kevinrue/snakemake_renv Call it a negative control, that doesn't cause the bug, even though the overall workflow is the same.

Hi Kevin, thanks for the detailed report.

Just a heads up, that when I run your initial example in a completely fresh way i.e. I just installed miniconda and then snakemake via conda, when I do

snakemake --use-conda --cores 2 renv # set up the R environment

I end up with:

Building DAG of jobs...
Creating conda environment workflow/envs/r.yaml...
Downloading and installing remote packages.
Environment for workflow/envs/r.yaml created (location: .snakemake/conda/6a474157)
Using shell: /usr/bin/bash
Provided cores: 2
Rules claiming more threads will be scaled down.
Singularity containers: ignored
Job counts:
    count   jobs
    1   renv
    1

[Tue Jul  7 11:21:32 2020]
rule renv:
    output: renv/activate.R
    jobid: 0

Activating conda environment: /tmp/snakemake_alevin_10x/.snakemake/conda/6a474157
Error: please call `renv::consent()` before using renv
Execution halted
[Tue Jul  7 11:21:34 2020]
Error in rule renv:
    jobid: 0
    output: renv/activate.R
    conda-env: /tmp/snakemake_alevin_10x/.snakemake/conda/6a474157

RuleException:
CalledProcessError in line 9 of /tmp/snakemake_alevin_10x/workflow/rules/other.smk:
Command 'source /home/mike/miniconda3/bin/activate '/tmp/snakemake_alevin_10x/.snakemake/conda/6a474157'; set -euo pipefail;  Rscript --vanilla /tmp/snakemake_alevin_10x/.snakemake/scripts/tmpex_mn5vw.renv.R' returned non-zero exit status 1.
  File "/home/mike/miniconda3/envs/kevin_rue/lib/python3.8/site-packages/snakemake/executors/__init__.py", line 2168, in run_wrapper
  File "/tmp/snakemake_alevin_10x/workflow/rules/other.smk", line 9, in __rule_renv
  File "/home/mike/miniconda3/envs/kevin_rue/lib/python3.8/site-packages/snakemake/executors/__init__.py", line 529, in _callback
  File "/home/mike/miniconda3/envs/kevin_rue/lib/python3.8/concurrent/futures/thread.py", line 57, in run
  File "/home/mike/miniconda3/envs/kevin_rue/lib/python3.8/site-packages/snakemake/executors/__init__.py", line 515, in cached_or_run
  File "/home/mike/miniconda3/envs/kevin_rue/lib/python3.8/site-packages/snakemake/executors/__init__.py", line 2199, in run_wrapper
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /tmp/snakemake_alevin_10x/.snakemake/log/2020-07-07T112107.033488.snakemake.log

I guess the imporant line is

Error: please call renv::consent() before using renv

It's pretty opaque to me what version of R I need to run that command in. Is it the system R that gets run? I don't see one installed in my conda environment where I installed snakemake.

Thanks for running the MWE already.

Error: please call renv::consent() before using renv

I did run into this and thought that I solved it, at least it wasn't asking me any more. I'll add a line to run renv::consent() in the pipeline so that you don't need to worry about it, and ping you back when it's done. If that works I'll just make a point to whoever uses the pipeline that by doing so they consent to renv.

It's pretty opaque to me what version of R I need to run that command in. Is it the system R that gets run? I don't see one installed in my conda environment where I installed snakemake.

That's also pretty strange. The point of the Conda environment (in your case /tmp/snakemake_alevin_10x/.snakemake/conda/6a474157) is to provide a new installation of R local to the project, specifically R 4.0.2 in this case, as indicated in my opening comment above. The following should give you the path to the local R binary:

conda activate .snakemake/conda/6a474157
which R

On a separate note, I have to take care of something else first today, but I would also like to put together a MWE that does not involve Snakemake, i.e., just Conda and renv. That may not be before tomorrow evening though.

You can pull from https://github.com/kevinrue/snakemake_alevin_10x branch renv again

I've added the following that should take care of consent

options(renv.consent = TRUE)

Great, it runs for me and on an Ubuntu system doesn't throw the error you're seeing. I just get:

...
Installing DelayedArray [0.14.0] ...
    OK [built from source]
Installing Rhdf5lib [1.10.0] ...
    OK [built from source]
Installing rhdf5 [2.32.2] ...
    OK [built from source]
...

So it seems like maybe it's a specific Mac / Conda issue. When I try to run the same example in a Mac Virtual machine I can't even get past the first step of the Rhdf5lib configure script. It errors with:

checking for gcc... x86_64-apple-darwin13.4.0-clang
checking whether the C compiler works... no    
configure: error: C compiler cannot create executables

It's fine if I use R and compilers installed outside of conda. I'm trying to understand why that would be the case, since it obviously gets further for you, but I guess it's something to do with it being in a VM.

checking for gcc... x86_64-apple-darwin13.4.0-clang
checking whether the C compiler works... no    
configure: error: C compiler cannot create executables

actually.. that's another error that I've come across on Mac indeed, though that one only happened once, I believe when I did things outside of Snakemake but within Conda. I haven't reproduced it yet.

For some reason in the Snakemake pipeline, as you mention, it does get further for me. It starts installing packages, compiling Rhdf5lib, and then crashes with the error message that I reported in my first comment.

I'll try to put together more MWE scenarios to help drill it down to the source of the issue. I'll report back here when I have that kind of additional information.

Here we go, it's not the original error that I reported, but this is the reproducible code to run into error configure: error: C compiler cannot create executables on my Mac

(base) kevin@x86_64-apple-darwin13 ~ % conda create --name r-env -c bioconda -c conda-forge -c defaults r-base=4.0.2 r-renv=0.11.0

[...
successful installation messages
...]

(base) kevin@x86_64-apple-darwin13 ~ % conda activate r-env

(r-env) kevin@x86_64-apple-darwin13 ~ % R

Then at the R prompt:

R version 4.0.2 (2020-06-22) -- "Taking Off Again"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin13.4.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> install.packages('BiocManager')

[...
successful installation messages
...]

> BiocManager::install('Rhdf5lib')
Bioconductor version 3.11 (BiocManager 1.30.10), R 4.0.2 (2020-06-22)
Installing package(s) 'BiocVersion', 'Rhdf5lib'
trying URL 'https://bioconductor.org/packages/3.11/bioc/src/contrib/BiocVersion_3.11.1.tar.gz'
Content type 'application/x-gzip' length 980 bytes
==================================================
downloaded 980 bytes

trying URL 'https://bioconductor.org/packages/3.11/bioc/src/contrib/Rhdf5lib_1.10.0.tar.gz'
Content type 'application/x-gzip' length 14438286 bytes (13.8 MB)
==================================================
downloaded 13.8 MB

* installing *source* package ‘BiocVersion’ ...
** using staged installation
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (BiocVersion)
* installing *source* package ‘Rhdf5lib’ ...
** using staged installation
checking for gcc... x86_64-apple-darwin13.4.0-clang
checking whether the C compiler works... no
configure: error: in `/private/var/folders/51/nz__2dx10yzd75rylr5cg2zc0000gq/T/RtmpyNjnxI/R.INSTALLfce633e16e0/Rhdf5lib':
configure: error: C compiler cannot create executables
See `config.log' for more details
ERROR: configuration failed for package ‘Rhdf5lib’
* removing ‘/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/Rhdf5lib’

The downloaded source packages are in
    ‘/private/var/folders/51/nz__2dx10yzd75rylr5cg2zc0000gq/T/RtmpRfn6Mn/downloaded_packages’
Updating HTML index of packages in '.Library'
Making 'packages.html' ... done
Warning message:
In install.packages(...) :
  installation of package ‘Rhdf5lib’ had non-zero exit status

I think both issues are inter related and revolve around the linker and the LDFLAGS being passed to is.

I think the error: C compiler cannot create executables is because the compiler and the linker are mismatched and it seems like the compiler asks for features the linker doesn't have. There's quite a few reports e.g. 1, 2, 3. I've got a temporary solution based on the first of those links and can get past that step, but I'm not sure how portable it is.

This let me reproduce your initial error, which is fixed if I make sure the LDFLAGS used to build R are also passed to the HDF5 compilation.

~I wanted to try this out in the full pipeline, but when I run the pipeline in my Mac VM I'm running into Error: Snakefile "Snakefile" not present. I'm sure I'm doing the same steps as when it worked on Ubuntu, but I lack familiarity with both Snakemake and OSX so maybe there's something obvious I'm missing.~ Scratch that, installing Snakemake via their instructions works.

Either way, you can try the patched version via:

BiocManager::install('grimbough/Rhdf5lib', ref = "ld-flags")

Hm... I may have to try again to be sure, but from the looks of it I ran into a different kind of error:

Bash

(base) kevin@x86_64-apple-darwin13 snakemake_alevin_10x % conda activate r-env
(r-env) kevin@x86_64-apple-darwin13 snakemake_alevin_10x % R

R version 4.0.2 (2020-06-22) -- "Taking Off Again"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin13.4.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> BiocManager::install('grimbough/Rhdf5lib', ref = "ld-flags")

Bioconductor version 3.11 (BiocManager 1.30.10), R 4.0.2 (2020-06-22)
Installing github package(s) 'grimbough/Rhdf5lib'
Using github PAT from envvar GITHUB_PAT
Downloading GitHub repo grimbough/Rhdf5lib@ld-flags
Running `R CMD build`...
* checking for file ‘/private/var/folders/51/nz__2dx10yzd75rylr5cg2zc0000gq/T/RtmpwEkf9i/remotes113d23349493/grimbough-Rhdf5lib-72c12df/DESCRIPTION’ ... OK
* preparing ‘Rhdf5lib’:
* checking DESCRIPTION meta-information ... OK
* cleaning src
* running ‘cleanup’
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
* building ‘Rhdf5lib_1.11.2-1.tar.gz’
* installing *source* package ‘Rhdf5lib’ ...
** using staged installation
checking for gcc... x86_64-apple-darwin13.4.0-clang

[...
lots of compilation messages 
...]

make[4]: Leaving directory '/private/var/folders/51/nz__2dx10yzd75rylr5cg2zc0000gq/T/Rtmp2bFzty/R.INSTALL1147d6666515d/Rhdf5lib/src/hdf5/hl/c++/src'
make[3]: Leaving directory '/private/var/folders/51/nz__2dx10yzd75rylr5cg2zc0000gq/T/Rtmp2bFzty/R.INSTALL1147d6666515d/Rhdf5lib/src/hdf5/hl/c++/src'
make[2]: Leaving directory '/private/var/folders/51/nz__2dx10yzd75rylr5cg2zc0000gq/T/Rtmp2bFzty/R.INSTALL1147d6666515d/Rhdf5lib/src/hdf5/hl/c++'
make[1]: Leaving directory '/private/var/folders/51/nz__2dx10yzd75rylr5cg2zc0000gq/T/Rtmp2bFzty/R.INSTALL1147d6666515d/Rhdf5lib/src/hdf5/hl'
configure:    HDF5_INCLUDE=hdf5/src
configure:    HDF5_CXX_INCLUDE=hdf5/c++/src
configure:    HDF5_HL_INCLUDE=hdf5/hl/src
configure:    HDF5_HL_CXX_INCLUDE=hdf5/hl/c++/src
configure:    HDF5_LIB=hdf5/src/.libs/libhdf5.a
configure:    HDF5_CXX_LIB=hdf5/c++/src/.libs/libhdf5_cpp.a
configure:    HDF5_HL_LIB=hdf5/hl/src/.libs/libhdf5_hl.a
configure:    HDF5_HL_CXX_LIB=hdf5/hl/c++/src/.libs/libhdf5_hl_cpp.a
configure:    SZIP_LIB=hdf5/szip/src/.libs/libsz.a
configure: creating ./config.status
config.status: creating src/Makevars
** libs
mkdir -p "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/src/"*.h "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/c++/src/"*.h "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/hl/src/"*.h "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/hl/c++/src/"*.h "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/src/"libhdf5.settings "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
mkdir -p "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp "hdf5/src/.libs/libhdf5.a" "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp "hdf5/c++/src/.libs/libhdf5_cpp.a" "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp "hdf5/hl/src/.libs/libhdf5_hl.a" "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp "hdf5/hl/c++/src/.libs/libhdf5_hl_cpp.a" "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp "hdf5/szip/src/.libs/libsz.a" "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
x86_64-apple-darwin13.4.0-clang -I"/Users/kevin/opt/anaconda3/envs/r-env/lib/R/include" -DNDEBUG -Ihdf5/src   -D_FORTIFY_SOURCE=2 -mmacosx-version-min=10.9 -isystem /Users/kevin/opt/anaconda3/envs/r-env/include -I/Users/kevin/opt/anaconda3/envs/r-env/include   -fPIC  -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/kevin/opt/anaconda3/envs/r-env/include -fdebug-prefix-map=/Users/runner/miniforge3/conda-bld/r-base_1593070335561/work=/usr/local/src/conda/r-base-4.0.2 -fdebug-prefix-map=/Users/kevin/opt/anaconda3/envs/r-env=/usr/local/src/conda-prefix  -c Rhdf5lib.c -o Rhdf5lib.o
x86_64-apple-darwin13.4.0-clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Users/kevin/opt/anaconda3/envs/r-env/lib/R/lib -Wl,-dead_strip_dylibs -Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-rpath,/Users/kevin/opt/anaconda3/envs/r-env/lib -L/Users/kevin/opt/anaconda3/envs/r-env/lib -o Rhdf5lib.dylib Rhdf5lib.o /Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/libhdf5.a /Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/libsz.a -lz -L/Users/kevin/opt/anaconda3/envs/r-env/lib/R/lib -lR -Wl,-framework -Wl,CoreFoundation
ld: unknown option: -platform_version
clang-10: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [/Users/kevin/opt/anaconda3/envs/r-env/lib/R/share/make/shlib.mk:6: Rhdf5lib.dylib] Error 1
ERROR: compilation failed for package ‘Rhdf5lib’
* removing ‘/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/Rhdf5lib’
Error: Failed to install 'Rhdf5lib' from GitHub:
  (converted from warning) installation of package ‘/var/folders/51/nz__2dx10yzd75rylr5cg2zc0000gq/T//RtmpwEkf9i/file113d234c0b54d/Rhdf5lib_1.11.2-1.tar.gz’ had non-zero exit status
In addition: Warning message:
In missing_devel_warning(pkgdir) :
  Package Rhdf5lib has compiled code, but no suitable compiler(s) were found. Installation will likely fail.
  Install XCode and make sure it works.

Let me know if you need the full log and I'll upload as a gist

Hmm, that ld: unknown option: -platform_version error is actually the cause behind the error: C compiler cannot create executables we were getting at the start of the script. But now we're getting it at the step where the package code is being linked, rather than the HDF5 library itself.

Do you get the same if you run pkgbuild::check_build_tools(debug = TRUE) in an R session?

In the same Conda environment r-env that I've been using above. Actually, I can't even get pkgbuild installed.

> install.packages('pkgbuild')
also installing the dependencies ‘ps’, ‘processx’, ‘assertthat’, ‘glue’, ‘fansi’, ‘backports’, ‘callr’, ‘cli’, ‘crayon’, ‘desc’, ‘prettyunits’, ‘R6’, ‘rprojroot’, ‘withr’
[...
downloads packages to install
...]
* installing *source* package ‘ps’ ...
** package ‘ps’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
x86_64-apple-darwin13.4.0-clang -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/kevin/opt/anaconda3/envs/r-env/include -fdebug-prefix-map=/Users/runner/miniforge3/conda-bld/r-base_1593070335561/work=/usr/local/src/conda/r-base-4.0.2 -fdebug-prefix-map=/Users/kevin/opt/anaconda3/envs/r-env=/usr/local/src/conda-prefix  -Wall px.c -o px
ld: unknown option: -platform_version
clang-10: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makevars:11: px] Error 1
ERROR: compilation failed for package ‘ps’
* removing ‘/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/ps’

Though the installation then carries on to try installing the other packages, succeeding for some that don't need compilation (e.g. crayon, prettyunits), but failing for some others (e.g. backports, fansi) with the same linker comand error.

I've ended up putting the line below in $HOME/.R/Makevars

LDFLAGS+=-mlinker-version=305

That seems sufficient to get most packages building in the conda activate r-env then R approach that I've mostly been testing with. There might be a more appropriate value than 305 (I think you can check ld -version) but this seems to at least work for me.

I'm then able to get to the successful end of an install via:

BiocManager::install('grimbough/Rhdf5lib', ref = "ld-flags")

Let me know if the above and/or the Xcode update changes the status for you.

I'm sure what I need to change in your workflow to make sure it installs that specific Github version of Rhdf5lib, so I'll leave that for now!

That worked! Namely Adding the following line in the $HOME/.R/Makevars

LDFLAGS+=-mlinker-version=305

And then running the following in the conda activate r-env

> BiocManager::install('grimbough/Rhdf5lib', ref = "ld-flags")
Bioconductor version 3.11 (BiocManager 1.30.10), R 4.0.2 (2020-06-22)
Installing github package(s) 'grimbough/Rhdf5lib'
Using github PAT from envvar GITHUB_PAT
Downloading GitHub repo grimbough/Rhdf5lib@ld-flags
Running `R CMD build`...
* checking for file ‘/private/var/folders/51/nz__2dx10yzd75rylr5cg2zc0000gq/T/RtmpjIt0ow/remotes12513595ed5f/grimbough-Rhdf5lib-d8b08e1/DESCRIPTION’ ... OK
* preparing ‘Rhdf5lib’:
* checking DESCRIPTION meta-information ... OK
* cleaning src
* running ‘cleanup’
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
* building ‘Rhdf5lib_1.11.2-1.tar.gz’
* installing *source* package ‘Rhdf5lib’ ...
** using staged installation
checking for gcc... x86_64-apple-darwin13.4.0-clang
checking whether the C compiler works... yes

[...
more messages
...]

** libs
mkdir -p "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/src/"*.h "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/c++/src/"*.h "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/hl/src/"*.h "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/hl/c++/src/"*.h "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/src/"libhdf5.settings "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
mkdir -p "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp "hdf5/src/.libs/libhdf5.a" "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp "hdf5/c++/src/.libs/libhdf5_cpp.a" "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp "hdf5/hl/src/.libs/libhdf5_hl.a" "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp "hdf5/hl/c++/src/.libs/libhdf5_hl_cpp.a" "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp "hdf5/szip/src/.libs/libsz.a" "/Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
x86_64-apple-darwin13.4.0-clang -I"/Users/kevin/opt/anaconda3/envs/r-env/lib/R/include" -DNDEBUG -Ihdf5/src   -D_FORTIFY_SOURCE=2 -mmacosx-version-min=10.9 -isystem /Users/kevin/opt/anaconda3/envs/r-env/include -I/Users/kevin/opt/anaconda3/envs/r-env/include   -fPIC  -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/kevin/opt/anaconda3/envs/r-env/include -fdebug-prefix-map=/Users/runner/miniforge3/conda-bld/r-base_1593070335561/work=/usr/local/src/conda/r-base-4.0.2 -fdebug-prefix-map=/Users/kevin/opt/anaconda3/envs/r-env=/usr/local/src/conda-prefix  -c Rhdf5lib.c -o Rhdf5lib.o
x86_64-apple-darwin13.4.0-clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Users/kevin/opt/anaconda3/envs/r-env/lib/R/lib -Wl,-dead_strip_dylibs -Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-rpath,/Users/kevin/opt/anaconda3/envs/r-env/lib -L/Users/kevin/opt/anaconda3/envs/r-env/lib -mlinker-version=305 -o Rhdf5lib.dylib Rhdf5lib.o /Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/libhdf5.a /Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/libsz.a -lz -L/Users/kevin/opt/anaconda3/envs/r-env/lib/R/lib -lR -Wl,-framework -Wl,CoreFoundation
ld: warning: -pie being ignored. It is only used when linking a main executable
installing to /Users/kevin/opt/anaconda3/envs/r-env/lib/R/library/00LOCK-Rhdf5lib/00new/Rhdf5lib/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (Rhdf5lib)

I guess now I just need to make sure i can reproduce this in the Snakemake workflow. Also, that's on Mac. Ultimately, I also need to make sure it runs on our CentOS cluster.

Thanks!

Excellent. I think the setting we've put in the Makevars is specific to this Mac ld, conda clang combination, I wouldn't expect you to encounter it on a CentOS machine. No guarantee that something totally new won't turn up though!

I'll merge the changes into the devel branch at Bioc and make sure it doesn't break anything there, and if all goes well I'll also patch the release branch since this is a problem introduced in that version.

Indeed, on CentOS, I've successfully run

BiocManager::install('grimbough/Rhdf5lib', ref = "ld-flags")

without even a $HOME/.R/Makevars file

grimbough / Rhdf5lib

Failure to install Rhdf5lib from source in a Conda installation of r-base 4.0.2 #32

Context

Issue

Related issues

Further notes