kestra-io / kestra

:zap: Workflow Automation Platform. Orchestrate & Schedule code in any language, run anywhere, 500+ plugins. Alternative to Zapier, Rundeck, Camunda, Airflow...
https://kestra.io
Apache License 2.0
13k stars 1.13k forks source link

Better display of topology (especially for the Dependencies) #5350

Open Skraye opened 1 month ago

Skraye commented 1 month ago

Feature description

Currently, the dependencies view is pretty great for simple use case, but when the graph become complex, it also become difficult to understand where the arrows goes : image

We need to try improving the display to avoid node being on arrows

joe-s1 commented 1 month ago

Thx for opening this, @Skraye!

sanketmagar2001 commented 1 month ago

Hi @Skraye, could you please clarify the expected outcome? We aim to ensure that there is always enough space between the nodes and arrows for clear visibility.

Skraye commented 1 month ago

@sanketmagar2001 the issue in the above graph is that some arrows are overlap by nodes, so it's not very readable which node is related to another

sanketmagar2001 commented 1 month ago

Hi @Skraye , thanks for the clarification! I understand the issue now. Let me know if you have any other suggestions or specific preferences regarding the layout!.

sanketmagar2001 commented 1 month ago

and do we have any examples flow , which I can use to test my changes on.

Skraye commented 1 month ago

@sanketmagar2001 I have no preference for the layout, the more readable, the better Here are some flow you can import, then you can go on the flow-a, pick the dependencies tab, and see that we can not see that flow-b also have a direct dependency with flow-c flow-topology-overlap.zip

sanketmagar2001 commented 1 month ago

Hi @Skraye , thank you for sharing the flow! I’ll work on it and come up with a solution. I’ll share my progress with you as soon as possible.

sanketmagar2001 commented 4 weeks ago

Hi @Skraye , @anna-geller , @MilosPaunovic , below is my understanding on the issue . please correct me if I am wrong 😀

Note : Here I am referring the example @Skraye has provided in above message.

We are currently on the Topology/Flow-A page, viewing the dependencies of the flow-a node.

Current Code Behavior:

1. Graph Request for Flow-A:****

When requesting the graph for flow-a (/flows/topology/flow-a/graph), we receive 3 nodes and 2 edges corresponding to the incoming and outgoing edges of flow-a. For instance, edges b → a and a → c.

2. Dependency Request for Flow-A:****

Similarly, the dependency request for flow-a (/flows/topology/flow-a/dependencies) returns the direct dependencies for flow-a, which are b → a and a → c.

Why we don’t see the edge b → c:

The current implementation only shows edges that are directly connected to flow-a (either incoming or outgoing). Therefore, b → c isn't displayed as it's not an edge directly linked to flow-a. However, when we expand flow-b or navigate to flow-b's dependency page, the edge b → c appears, which is the intended behavior. It ensures that only the dependencies of the node we're focusing on (in this case, flow-a) are displayed.

The Issue: This example does not accurately represent the issue we are facing with edge conflicts in more complex graphs. Specifically, the issue arises when multiple arrows (edges) overlap or conflict with nodes, leading to visual clutter in more intricate topologies.

Screenshot 2024-10-20 at 2 40 42 PM

Proposed Solution:

1) Change the Edge Type to Bezier:

Currently, we are using the smoothstep edge type provided by Vue Flow, but the default edge type in the @vue-flowlibrary is bezier. Example from the @vue-flow library:

export type SmoothStepEdgeType<Data = ElementData, CustomEvents extends Record<string, CustomEvent> = any> = DefaultEdge<
  Data,
  CustomEvents
> & {
  type: 'smoothstep'
  pathOptions?: SmoothStepPathOptions
}

export interface BezierPathOptions {
  curvature?: number
}

export type BezierEdgeType<Data = ElementData, CustomEvents extends Record<string, CustomEvent> = any> = DefaultEdge<
  Data,
  CustomEvents
> & {
  type: 'default'
  pathOptions?: BezierPathOptions
}

Switching the edge type from smoothstep to default (which uses a bezier curve) would improve readability in complex graphs and reduce conflicts with nodes.

Here's an example of bezier curve arrows in a complex graph: example graph with Bézier curve edge type

Additionally, we could implement a feature that highlights the routes of a selected node, making it easier to follow connections.

type : smoothstep

Screenshot 2024-10-20 at 8 31 16 PM

type : bezier

Screenshot 2024-10-20 at 7 56 15 PM

2) Dynamically Calculate Edge Curvature and Offset:

We can introduce dynamic calculation of edge curvature and offset based on the distance between nodes. These parameters will allow the edge to deviate from a straight line and move further away from nodes, preventing overlaps and visual clutter, especially in dense graphs.

Curvature: Controls the bend of the edge. Offset: Defines the distance of the curve from the nodes.

Here’s an example of how we could pass these options to the edge configuration in Vue Flow:

{
  source: 'node-1',
  target: 'node-2',
  type: 'smoothstep',
  pathOptions: {
    curvature: 0.5,  // Mid-range curve
    offset: 40,      // Moves the curve away from the nodes
  },
}

Currently, these values are hardcoded, but we can calculate them dynamically based on the positions of the source and target nodes to ensure proper spacing and prevent node overlap.

Skraye commented 4 weeks ago

Hey @sanketmagar2001, thank you for your feedback, very complete!

You're right. My example is too simple, and you need to expand B to see the issue. Nice catch. Seeing both solutions, I think we should evaluate the first one. The second adds more complexity to something that is already pretty heavy.

Additionally, we could implement a feature that highlights the routes of a selected node, making it easier to follow connections.

It is something we currently already have, see the white highlight around nodes : image

Would you mind making a PR with the Bezier edge type, so that on our side we can try it with different flows, to verify it does not have side effects?

MilosPaunovic commented 4 weeks ago

Just a note that making a PR here might not be that straight-forward, as we're generating topology view using our different repository, https://github.com/kestra-io/ui-libs.

If you can't make sense with the setup @sanketmagar2001, feel free to reach out.

sanketmagar2001 commented 4 weeks ago

Hi @MilosPaunovic @Skraye , just to clarify, would the changes need to be made on both sides? Specifically, in the flow dependencies view (/kestra/ui/src/components/flows/FlowDependencies.vue) and in generating the topology view (https://github.com/kestra-io/ui-libs/blob/master/src/components/topology/Topology.vue)?

Skraye commented 4 weeks ago

Hey @sanketmagar2001 , I believe they use both ui-libs behind, so change would be make to both

sanketmagar2001 commented 3 weeks ago

ok , will create the PR