Convert to renv

[russHyde]

fixes #29

Initially had a conda environment (plus some install_githubs)

Had trouble updating the env: both slowness of solving env, and impossibility
of installing some packages (pkgnet)

The initial env was based on R-3.5 and had rstudio inside it

So it was a reluctant beast

Also needed to have a separate ./setup script to ensure that github packages
were installed correctly, once the conda-env was in place.

Recommendation

• Remove r-base & rstudio from the existing environment
  • Then saved new env description to envs/{environment.yml,requirements.txt}
• Install r-base (>=4) into the env, with 'renv', 'rstudioapi'
  • conda install r-base r-renv r-rstudioapi would only install r-base=3.6.1
  • conda install r-base=4.0.3 r-renv r-rstudioapi threw conflicts
  • conda update did nothing
  • conda env create --file envs/environment.yml --name code-as-data ran fine - but installing r-base r-renv and r-rstudioapi on that failed
  • therefore: -
  • removed "code-as-data" env
  • made new empty conda env "code-as-data"
  • installed main programs from original "code_as_data" and the required R
  • conda install -c conda-forge -c bioconda \ snakemake=5.10.0 nextflow=19.10.0 r-base=4.0.3 r-renv r-rstudioapi - Run renv::init
• Install all remotes (cloc, gitsum, dupree-dev) using remotes::install_gitsum - Remove the script for installing github remotes into the conda env
  • and remove the config section for remote-package installs - Run each script in turn: installing anything that is missing into the renv - Run renv::snapshot
• Update README and ./setup: how to setup an identical working env

TODO:

• Bump to recent versions of snakemake / nextflow
• [ ] Check the workflow_tools pipelines all still work
  • [x] snakemake subworkflow fixed (by linking to renv/.Rprofile in root of project)
  • [ ] nextflow: problems (see below)
• [x] rename ./envs as ./conda
• Add git-hook to check that renv::snapshot has been run

[russHyde]

Attempt to get nextflow pipeline to run with renv was a failure.

Since nextflow processes run in a temp directory, they don't necessarily have access to files that are implicitly required.

Here,

• we are using renv to define the dependencies for the project
• we don't need .Rprofile, renv/ in the command-line call to run an R script
• but if they are not available in the working directory when that R script is ran...
• ... any dependencies defined by the renv will be unavailable to that script

So, we need a way to copy / link implicit file-dependencies into the working directory for any process that runs an R script; but I can't find a good explanation for how to do that within nextflow.

[ekatsevi]

I think I resolved this issue. All you need for an R script to use the renv packages is to have the R_LIBS_USER environment variable pointing to the underlying R library path. You can get this path in R by activating the renv and then typing cat(paste0(.libPaths(), collapse = ":")). According to the Nextflow documentation on configuration, "The env scope allows the definition one or more variable that will be exported in the environment where the workflow tasks will be executed." Therefore, if your Nextflow configuration file contains the line env.R_LIBS_USER = /path/to/renv/libs, then each process will have access to your renv libraries. You can even carry out the above steps programmatically in a bash script as follows:

RENV_R_LIBS_USER=$(Rscript -e '
renv::activate(); 
cat(paste0(.libPaths(), collapse = ":"))')
echo "env.R_LIBS_USER = \"$RENV_R_LIBS_USER\"" > nextflow.config

Hope this helps!

[russHyde]

Amazing. I'd recently started looking back at this project (though I don't use nextflow for work nowadays). Thanks for posting your solution.