search

kedro-org / kedro-viz

Visualise your Kedro data and machine-learning pipelines and track your experiments.
https://demo.kedro.org
Apache License 2.0
672 stars 110 forks source link

Refactor Modular Pipelines in the Kedro-viz backend #1899

Closed rashidakanchwala closed 3 months ago

rashidakanchwala commented 4 months ago

Description

The logic of how we calculate modular pipelines in Kedro-viz backend differs from how it is computed in Kedro. For instance, modular pipelines are not associated with namespaces, and neither are the datasets in a modular pipeline. Only nodes have namespaces, and based on the way the datasets are consumed by different nodes, these datasets can be regarded as input/output/children to each modular pipeline.

This PR is to revise the current logic and make it more like the way Kedro does it. Modular Pipelines have caused a lot of issues for the user based on the difference in approach. With this refactor, we hope to align the approaches.

Context

Why is this change important to you? How would you use it? How can it benefit other users?

Possible Implementation

(Optional) Suggest an idea for implementing the addition or change.

Possible Alternatives

(Optional) Describe any alternative solutions or features you've considered.

Checklist

yury-fedotov commented 4 months ago

Is this issue potentially resolving that one? Just curious.

rashidakanchwala commented 4 months ago

Yes, it does :) - the PR is also pretty much done, https://github.com/kedro-org/kedro-viz/pull/1897