MI-DPLA / combine

Combine /kämˌbīn/ - Metadata Aggregator Platform
MIT License
26 stars 11 forks source link

normalize get_lineage, get_jobs_lineage, get_downstream_lineage #322

Closed ghukill closed 5 years ago

ghukill commented 5 years ago

Lineage has evolved some, and would make sense to have methods that were more like:

ghukill commented 5 years ago

Now that it's possible to edit input jobs via methods (https://github.com/WSULib/combine/issues/325), puts pressure to have the re-run lineage a bit smarter than id sorting.

For example, with Job 59 added after the fact as input job to Job 58, re-running 59 with downstream still has 58 re-run first (because of lower id), when in fact 59 should re-run first:

screen shot 2018-10-11 at 3 12 59 pm

ghukill commented 5 years ago

Working to handle a situation like this:

screen shot 2018-10-12 at 10 10 56 am

Where re-run order should be: A,D,C,B,E

ghukill commented 5 years ago

Addressed using toposort. Handles circular dependencies -- input jobs -- for get_downstream_jobs(), but not yet for get_lineage(). Will want to include there as well.

ghukill commented 5 years ago

Normalized a bit. Closing here, with intent of opening more specific issues.