richfitz / remake

Make-like declarative workflows in R
Other
340 stars 32 forks source link

dependencies for specific tasks #12

Closed mwpennell closed 9 years ago

mwpennell commented 9 years ago

this may not be important but one issue to consider is whether dependencies should only be required for the tasks for which they are needed.

for example, in the model adequacy project we may want to build the dataset

m <- maker$new()
m$make("data")

however, this requires that the packages gridExtra be installed to run even though this package is only used for the plotting functions downstream and not required to actually build the data. not sure how to fix it but it might be a nice feature so that the workflow is more modularized.

richfitz commented 9 years ago

Yeah, this is definitely something needed. It's been bugging me to (xlsx package, looking at you) but wanted to make sure that it annoyed someone else before implementing it. I have a big list of this sort of stuff that would just end up as @richfitz-specific bloat if I did it all!

So within the maker file we could have:

targets:
  mytarget:
    command: whatever()
    packages: [foo, bar]

that would load packages foo and bar before running whatever(). I think it would be important to unload the packages after use though, because otherwise later on tasks could accidentally depend on them and lead to hard to trace errors where things worked depending on what was built before it.

It's a bit tricky in the case of fake targets though. Suppose you have

targets:
  grouping_target:
    depends: [a, b, c]
    packages: foo
  a:
    command: something_that_needs_foo()
  ...

Then running m$make("grouping") would work, but m$make("a") would fail.