kedro-org / kedro-viz

Visualise your Kedro data and machine-learning pipelines and track your experiments.
https://demo.kedro.org
Apache License 2.0
646 stars 106 forks source link

Refactor flowchart models for modular pipelines change #1937

Closed ravi-kumar-pilla closed 13 hours ago

ravi-kumar-pilla commented 3 weeks ago

Description

Partially resolves - #1899

Development notes

QA notes

Checklist

ravi-kumar-pilla commented 1 week ago

In the description you mention

Removes namespace property from GraphNode and retain the property only for TaskNode following Kedro Framework

and I see a lot of methods related to namespaces have been removed. Can you explain why that change was needed and how namespaces are now set and used in Viz?

So earlier we used to have namespaces at the GraphNode level (which includes TaskNode, DataNode and others). After having discussion with Ivan, we realized datasets were never a part of a namespace and to have the same schema as Kedro, we removed the namespace from GraphNode and retained it to TaskNode.

For a TaskNode namespace is set as before via kedro object like -

    @field_validator("namespace")
    @classmethod
    def set_namespace(cls, _, info: ValidationInfo):
        return info.data["kedro_obj"].namespace

For other nodes, we do not need namespace. Lot of methods related to namespace like _expand_namespaces, _get_namespace in GraphNode are not required as they were used to calculate the modular_pipelines each node belongs to . Since we shifted the logic ( as you mentioned here ) before creating the nodes, we do not need them now.

Thank you