Closed ghukill closed 6 years ago
Actually, this may not be as bad as anticipated. Looking more closely, it would appear that stages "jumping" in and out, are actually duplicates (this can be seen in the first image). It does actually trace up and downstream, but does not differentiate for duplicates.
Addressed "jumping" of stages by using OrderedDict
to remove dupes, but retain ordering.
However, the method Record.get_record_stages()
now has optional remove_duplicates
argument that can be used to retain duplicates if set to False
, helpful for generating lineage much like jobs. So that should be possible as well (keeping this issue open).
Fixed, mostly, by utilizing combine_id
to show lineage for records. These kind of impossible hops are gone. Closing.
As it stands now, there is a "Stages" section for Record details that show all the upstream stages of a record. Here's a record at a late stage, involved in many jobs:
Here is the same record at an earlier stage, showing less downstream:
There are two problems here:
Graph for Record stages
A graph similar to the job lineage graph might be a neat way to show the record stages, and how they relate. Same interaction with the table of highlighting upstream only.
Stages disappearing
Certainly doable, but would it make sense?