[FEATURE] Plotting task graphs

[rdmolony]

Is your feature request related to a problem? Please describe. I find task execution graphs really useful in communicating workflows to non-technical audiences, specifically they are interested in exactly what is being performed on the data to get it from A to B. However, this can get complex fast for distributed workflows!

dask:

prefect:

The dask graph is really hard to follow!

Describe the solution you'd like The dask graph is for the nerds, I'd like a higher-level prefect style graph instead for non-tech audiences and documentation. prefect creates uses graphviz to create nodes for each task in its task graph and then plot them (I'm slowly learning @kvnkho :) - https://github.com/PrefectHQ/prefect/blob/5de58efaba956b431335d99acab07eaf6a362e1b/src/prefect/core/flow.py#L1290) which I suspect could be adapted here

Describe alternatives you've considered It's already possible to leverage the dask graph by writing a dask function and using the dask execution engine, and so perhaps there is no need to do this natively!

[rdmolony]

Vision: Interactive vector based task graph Easiest: graphviz - pixel based

Resources:

• bokeh - https://docs.bokeh.org/en/latest/docs/user_guide/graph.html
• plotly - https://github.com/BusinessOptics/dash_interactive_graphviz

FugueWorkflow.visualize() has to happen on local

[rdmolony]

Apologies team, I haven't made time to work on this.

I spotted Mermaidjs which reminded me of this PR, though I imagine other Python libs have since worked out a non graphviz solution since https://github.com/mermaid-js/mermaid

Other:

• d3 https://github.com/ploomber/ploomber/blob/master/src/ploomber/dag/plot.py