search

kubeflow / pipelines

Machine Learning Pipelines for Kubeflow
https://www.kubeflow.org/docs/components/pipelines/
Apache License 2.0
3.52k stars 1.59k forks source link

Best data versioning tool/framework with kfp for seamless integration. #7356

Closed vamshi-rvk closed 2 years ago

vamshi-rvk commented 2 years ago

Hi, our team has started implementing kfp for mlops, and we are researching on best methods to data version. We primarily work on images but not limited to it. Which would be the best data versioning framework which works along with kfp with seamless integration and featureset.

zijianjoy commented 2 years ago

/assign @connor-mccarthy

connor-mccarthy commented 2 years ago

@vamshi-rvk, thanks for your question.

kfp does not natively integrate with any data versioning tools/frameworks, but dvc is a popular open-source data version control system used in many MLOps workflows, including those that work with image data. I recommend you see if this service meets your needs.

vamshi-rvk commented 2 years ago

@vamshi-rvk, thanks for your question.

kfp does not natively integrate with any data versioning tools/frameworks, but dvc is a popular open-source data version control system used in many MLOps workflows, including those that work with image data. I recommend you see if this service meets your needs.

thanks for the respons.