cylc / cylc-flow

Cylc: a workflow engine for cycling systems.
https://cylc.github.io
GNU General Public License v3.0
327 stars 93 forks source link

graph traversal utility #6237

Open hjoliver opened 2 months ago

hjoliver commented 2 months ago

See discussion at #6221 - we need a way for users to easily see what happens downstream of arbitrary points in the graph, particularly prior to staging a runtime intervention.

static

Plot all paths downstream of chosen task(s) in the graph, e.g.:

$ cylc graph [--from-task=4/model] [--max-edges=10] [--stop-cycle=5]

live (GUI)

Plot all paths downstream of chosen task(s) in the graph, as above, but get the result into the datastore somehow.


notes

oliver-sanders commented 1 week ago

I think this is potentially a good idea, it might be a useful tool in workflow development and testing amongst other things.

I think some SoD interventions are going to push us towards developing some graph traversal functionality anyway. E.G. the proposed group trigger functionality will need to be able to identify start tasks within the group. So that might play into the development of a cylc graph command.

From the perspective of helping users to perform interventions, this would help, bit I think it is more important for us to develop simpler interventions that do not require graph inspection in the first place. That's not to say this isn't a potentially useful tool, but users shouldn't have to go charting a course through the graph to perform targetted interventions. The group-trigger proposal will hopefully avoid the need for users to identify the upstream tasks (to set/trigger) and the downstream tasks (to hold/remove) when re-running subgraphs of tasks (and may also avoid some bugbears around parentless tasks) so is a preferable solution IMO.