search

GEO-BON / bon-in-a-box-pipelines

BON in a Box 2.0 - Sampling optimisation and indicator pipelines
MIT License
10 stars 7 forks source link

freeze dependencies version #108

Open tpoisot opened 11 months ago

tpoisot commented 11 months ago

This

https://github.com/GEO-BON/biab-2.0/blob/dabd754c98def5c2288dbe07eefe036344666558/runners/julia-dockerfile#L4

will install the latest versions of the packages -- wouldn't it be safer to use an explicit compat block to make sure we known which version will run in practice?

gottacatchenall commented 11 months ago

Yeah probably

gottacatchenall commented 11 months ago

I've got a version of this that builds the julia docker runner based on a Project toml, I'll push tomorrow

gottacatchenall commented 11 months ago

PR here #114 , it might be smart to use separate environments for each script, or else we may run into issues with a single global environment later on add more julia scripts get added

simonrolph commented 8 months ago

Hi, I was going to make an issue about the same issue in relation to R, but I am adding a comment in here as the issue doesn't specify to be julia specific.

The same issue applies for R. For example in https://github.com/GEO-BON/bon-in-a-box-pipelines/blob/main/scripts/SDM/consensusPredictions.R this code

## Install required packages
packages <- c("terra", "rjson", "raster", "dplyr", "dismo")
new.packages <- packages[!(packages %in% installed.packages()[,"Package"])]
if(length(new.packages)) install.packages(new.packages)

## Load required packages

library("terra")
library("rjson")
library("raster")
library("dplyr")
library("dismo")

is not reproducible in the medium/long term as no package versions are defined, as soon as a package updates with a change then the pipeline won't work. I'm not sure if the version of the package used in each pipeline run is recorded anywhere?

In R the adopted way of maintain libraries is to use renv https://rstudio.github.io/renv/articles/renv.html but I have no idea how this might be integrated into BiaB

There is the install_versions R function in {remotes} https://rdrr.io/cran/remotes/man/install_version.html which means you can specify a version of an R package. Requires the remotes package to be installed, could be added to the r runner?

jmlord commented 8 months ago

Linked to milestones for the new year: https://github.com/GEO-BON/bon-in-a-box-pipeline-engine/milestone/3 https://github.com/GEO-BON/bon-in-a-box-pipelines/milestone/2