Implement clean and rebuild functions

Once the pipeline has been executed, it stores information on the locations of all targets, dependencies and source files on disk. It also stores each recipe as a string.

We can leverage this to implement clean and rebuild functions:

clean_pipeline() would delete all targets
build_pipeline() would execute make_with_source() or make_with_recipe() for each source/recipe in the pipeline in the appropriate order
rebuild_pipeline() would call clean_pipeline() and then build_pipeline()

The most challenging part of this will be figuring out what the 'appropriate order' of execution is. GNU make decides where to start using topological sorting. Possibly there's an algorithm in visNetwork that I can use. Otherwise, I'll need to code something up.

Once this is done, it might be neat to introduce a pipeline() function for defining a pipeline in one fell-swoop

my_pipeline <- Pipeline$new(
  list(
    dependencies = c("data/0_raw_data.csv", "lookup/concordance.csv"),
    source = c("1 data_prep.R"),
    targets = c("data/1_data.Rds")
  ),
  list(
      dependencies = c("data/1_data.Rds", "data/0_pop.Rds"),
      recipe = {
        dat <- readRDS("data/raw_data.Rds")
        pop <- readRDS("data/pop_data.Rds")
        merged_dat <- merge(dat, pop, by = "id")
        saveRDS(merged_dat, "data/2_data.Rds")
      },
      targets = c("data/2_data.Rds")
  )
)

rebuild_pipeline(my_pipeline)

kinto-b / makepipe

Implement clean and rebuild functions #3