Closed ghukill closed 5 years ago
Now that it's possible to edit input jobs via methods (https://github.com/WSULib/combine/issues/325), puts pressure to have the re-run lineage a bit smarter than id
sorting.
For example, with Job 59
added after the fact as input job to Job 58
, re-running 59
with downstream still has 58
re-run first (because of lower id
), when in fact 59
should re-run first:
Working to handle a situation like this:
Where re-run order should be: A,D,C,B,E
Addressed using toposort. Handles circular dependencies -- input jobs -- for get_downstream_jobs()
, but not yet for get_lineage()
. Will want to include there as well.
Normalized a bit. Closing here, with intent of opening more specific issues.
Lineage has evolved some, and would make sense to have methods that were more like:
get_lineage(upstream=False, downstream=True)
True
, would be a "web" of connected Jobs