how to build from scratch

[vjcitn]

1) use ubuntu 18.04 2) use endow_ubuntu() in BiocBBSpack [NOT: obtain the mystring.txt from BiocBBSpack/inst/ubuntu] and run as sudo to set up the linux infrastructure, takes about 30 minutes and two interactions on ttf EULA and a service restart are needed 3) install libsbml from source 4) svn co/configure/build/install R-devel from source 5) in R, install.packages(c("BiocManager", "remotes",)) 6) in R: BiocManager::install(c("vjcitn/pkgbuild", "vjcitn/BiocBBSpack")) 7) in R: BiocManager::install(BiocBBSpack::installed_r, Ncpus=12) 8) in a writeable folder, run

library(BiocBBSpack) # ensure that the version of pkgbuild is the fork at github.com/vjcitn
jnk = sapply(bioc_software_packagelist(), getpk)  # could be parallelized? takes 1.5h serially in jetstream

9) in a different writeable folder, run

library(BiocBBSpack) # ensure that the version of pkgbuild is the fork at github.com/vjcitn
num_cores = [your choice]
library(pkgbuild)
library(BiocBBSpack)
cands = list_packs_to_update("../bioc_sources", ".") # assume ../bioc_sources is where you did part 8)
library(parallel)
options(mc.cores=num_cores)
chk = mclapply(cands, function(x) {Sys.sleep(runif(1, 2, 6)); try(build1(x, "."))})

On jetstream this seems to do about one package per core every 2 minutes ... so about 3h with 20 cores to do 1800 packages

Notes: 7) will install over 2200 R packages as of July 7 2019, in approximately 2 hours

with that, these will fail: ‘cairoDevice’, ‘lpsymphony’, ‘gWidgetsRGtk2’, ‘IHW’, ‘rsbml’, ‘charmData’

lpsymphony fails because it checks R CMD config F77 instead of R CMD config FC which is required for R-devel -- when the source is altered by hand to use "FC" in the configure script, lpsymphony and IHW will install

rsbml failure may have to do with additional linux infrastructure needed -- but even after installing libsbml5/-dev

Error: package or namespace load failed for ‘rsbml’: .onLoad failed in loadNamespace() for 'rsbml', details: call: dyn.load(file, DLLpath = DLLpath, ...) error: unable to load shared object '/usr/local/lib/R/library/00LOCK-rsbml/00new/rsbml/libs/rsbml.so': /usr/local/lib/libsbml.so.5: undefined symbol: _ZN9Validator13addConstraintEP11VConstraint Error: loading failed Execution halted

[vjcitn]

July 7 2019, running:

# crude way of building bioc_tarballs from bioc_sources using R
options(error=recover, warn=2)
num_cores = 20
library(pkgbuild)
library(BiocBBSpack)
cands = list_packs_to_update("../bioc_sources", ".")
library(parallel)
options(mc.cores=num_cores)
chk = mclapply(cands, function(x) {Sys.sleep(runif(1, 2, 6)); try(build1(x, "."))})
save(chk, file="../chk.rda")

leads to 1526 tarballs and 200 try-errors recorded in chk.rda. Time elapsed 3h17m on XSEDE jetstream (TACC) s1.xlarge (CPU: 24, Mem: 60 GB, Disk: 240 GB, Disk: 240 GB root), no volumes attached.

[vjcitn]

A simple second attempt produces tarballs for an additional 46 packages. This needs to be explained -- could be resource contention or undeclared interdependencies...