kinto-b / makepipe

Tools for constructing simple make-like pipelines in R.
https://kinto-b.github.io/makepipe/
GNU General Public License v3.0
30 stars 0 forks source link

Add pipeline initialisation helper #39

Closed kinto-b closed 2 years ago

kinto-b commented 2 years ago

As a first approximation, we expect the dependencies to be anything read in with readRDS(), read_*() and read.*() and the targets to be anything written out with saveRDS, write_*() or write.*()

So it should be possible to do static analysis of the scripts in a directory to pull out the likely dependencies and targets and produce a pipeline consisting of multiple make_with blocks.

kinto-b commented 2 years ago

also source() for deps

kinto-b commented 2 years ago

One possible API:

initialise_pipeline <- function(files = NULL, in_funcs = c("source", "readRDS", "read\\..**", "read_.*"), out_funcs = c("saveRDS", "write\\..*", "write_.*")) {
  if(is.null(files)) files <- list.files(".", pattern = "\\.R$", ignore.case = TRUE)
  ...
} 

Or

initialise_pipeline <- function(files = NULL, in_funcs = NULL, out_funcs = NULL) {
  if(is.null(files)) files <- list.files(".", pattern = "\\.R$", ignore.case = TRUE)
  if(is.null(in_funcs)) in_funcs <- c("source", "readRDS", "read\\..**", "read_.*")
  if(is.null(out_funcs)) out_funcs<- c("saveRDS", "write\\..*", "write_.*")
  ...
} 
kinto-b commented 2 years ago

Hm, on second thought not so easy since we might have:

fp <- "my/file/path.Rds"
readRDS(fp)