kedro-org / kedro-viz

Visualise your Kedro data and machine-learning pipelines and track your experiments.
https://demo.kedro.org
Apache License 2.0
679 stars 112 forks source link

Kedro-Viz Competitor Benchmark #1750

Closed NeroOkwa closed 1 month ago

NeroOkwa commented 9 months ago

Description

The goal of this research is to benchmark Kedro-Viz features with those of alternative/complementary tools (dbt, MLflow, Prefect, Great Expectation, Dagster, Metaflow, DVC) or those usually grouped in a similar category with Kedro (ML pipeline visualisation, orchestrators).

The emphasis would be on the visualisation/UI layer of these tools, towards identifying areas of improvement for Kedro-Viz.

Context

Why is this change important to you? How would you use it? How can it benefit other users? The research would seek to understand:

  1. What problem were they trying to solve for their users ?
  2. How did they solve it and what was the effectiveness of this solution?
  3. Is this a problem for kedro users? Have kedro users asked for this ? has it come up in user research?
  4. How has Kedro tried to solve it ? was the solution effective ? is this a missing functionality for Kedro?

Possible Implementation

Research presentation, and prioritised list of features for improvement.

Outcome

Recording, Deck, Notes

Below is the prioritised list of features benchmarked with Kedro-Viz.

Feature Is this on Kedro-Viz ? Affects Kedro users? Valuable to Kedro users ? Will impact adoption ? Decision Type Github link Status
Metaflow - Card Yes (with Vizro integrated with Kedro) Yes Yes Yes Yes (plugins) Big bet #1457
dbt - Filtering/syntax selection, Dagster graph filtering Yes - in side panel but very limited Yes Yes Yes Yes Big bet #1784 Ongoing
Dagster - Re-materialise/backfill data from the graph - Run node/pipeline No Yes Yes Yes Yes (ability to run node/pipeline, parameter editor from the UI) Big bet #1753 #1279
Dagster - Re-materialise/backfill data from the graph Yes - autoreload flag (but not on UI) Yes Yes Maybe Yes (improve autoreload time and consider showing it in the flowchart) Improve #1751 #1704
Metaflow - Navigation bar/path specific URL No Yes Yes Yes Yes Improve #1736 Ongoing
dbt - Docs + Lineage graph No Yes Yes Yes Yes Improve
Dagster - Tags and icons Yes Yes Yes No (engagement) Yes (improve existing feature) Improve #1148
dbt - Color coding nodes No Yes Yes. Only if combined with better filtering and layers No (engagement) Maybe Improve
Dagster - Asset groups, expand/collapse all Yes Yes Yes Maybe Maybe (improve existing feature) Improve #1752 Ongoing
DVC - Experiment tracking using GIT No Yes - 1 user for kedro-mlflow Yes Yes No N/A
Dagster - Left side bar and search field Yes - sidebar Yes-search field No / No Yes / Yes (improving) No/ No (engagement) No N/A
astrojuanlu commented 1 month ago

Moved this to https://github.com/kedro-org/kedro-viz/wiki/Competitors-benchmark-2024%E2%80%9002