search

kedro-org / kedro-devrel

Kedro developer relations team use this for content creation ideation and execution
Apache License 2.0
0 stars 3 forks source link

Comparative content to clarify Kedro positioning #31

Closed stichbury closed 1 year ago

stichbury commented 1 year ago

What are the alternatives to using Kedro?

I took the following from https://blog.streamlit.io/using-chatgpt-to-build-a-kedro-ml-pipeline/

Airflow: Airflow is a platform to programmatically author, schedule, and monitor workflows. It was originally developed by Airbnb, and is now a popular open-source project. Airflow provides a flexible and powerful platform for building and managing ML pipelines, and has a large user base and community.

Kubeflow: Kubeflow is an open-source project that aims to make it easy to deploy and manage machine learning workflows on Kubernetes. It provides a range of tools and frameworks for building, deploying, and managing ML pipelines, including Jupyter notebooks, TensorFlow, and PyTorch.

Prefect: Prefect is an open-source platform for building, scheduling, and monitoring data pipelines. It is designed to be highly scalable and resilient, and provides a range of features for building and managing ML pipelines, including support for distributed execution and automatic retries.

Luigi: Luigi is an open-source Python library for building complex pipelines of batch jobs. It was developed by Spotify and is designed to be easy to use and flexible. Luigi provides a range of features for building and managing ML pipelines, including support for dependencies, parallel execution, and error handling.

These are just a few examples of the alternatives to Kedro that you can consider. Each platform has its own strengths and weaknesses, and the right choice will depend on your specific needs and requirements.

I think we should create some content (either a non-navigable "topic" page or a blog post) that compares Kedro with each of these and maybe provides a visual "matrix" to check off which each offers and compare with Kedro (maybe a bit like this).

stichbury commented 1 year ago

This is a duplicate of https://github.com/kedro-org/kedro/issues/3012