ropensci / targets

Function-oriented Make-like declarative workflows for R
https://docs.ropensci.org/targets/
Other
919 stars 74 forks source link

error = "trim" #1331

Closed wlandau closed 2 weeks ago

wlandau commented 2 weeks ago

Prework

Proposal

Suggested by @hadley in https://github.com/ropensci/targets/discussions/1310#discussioncomment-10205110: if a target fails, then

  1. Don't start anything downstream of that target.
  2. Cancel dynamic branches with the same parent.
  3. Let everything else run.

For (2), there is the question of what we mean by "cancel". When one branch throws an error, some of its siblings may not have started yet, whereas others may already be running. I would prefer to cancel the former and let the latter keep running. Reasons:

  1. It seems friendlier for users and easier for them to understand.
  2. In distributed computing scenarios, it won't require https://github.com/shikokuchuo/mirai/issues/112 or https://github.com/HenrikBengtsson/future/issues/93. (NB clustermq doesn't support cancellation either AFAIK.)
  3. It's much easier to implement and maintain.
  4. It's more consistent with the rest of targets: error = "trim" will treat currently running targets the exact same way as error = "abridge"

Overall, I think error = "trim" will be a nice addition to targets.

wlandau commented 2 weeks ago

Added in #1332