tolgakaratas
commented
1 year ago I forgot to put this links : https://dvc.org/doc/start/experiments/experiment-tracking
https://cml.dev/doc/cml-with-dvc
https://mlem.ai/doc/get-started https://mlem.ai/doc/gto/get-started
https://marketplace.visualstudio.com/items?itemName=Iterative.dvc https://dvc.org/doc/dvclive/how-it-works https://dvc.org/doc/start/data-management/metrics-parameters-plots https://dvc.org/doc/start
You would like to try these: https://github.com/iterative/cml#getting-started
Continuous Machine Learning (CML) is CI/CD for Machine Learning Projects https://cml.dev/#use-cases
https://gitlab.com/iterative.ai/cml-base-case
https://github.com/iterative/cml/blob/master/.gitlab-ci.yml
https://github.com/iterative/cml https://www.youtube.com/@dvcorg8370
What is DVC?
đź”— Git for Data (What is DVC?)
Data Version Control (DVC) is an open-source tool for data science and machine learning teams to manage datasets, ML models, and experiments in Git. Key parts include:
“Git for Data and Models” - DVC extends Git versioning to large files like datasets and ML models for rigorous project management. Use your regular Git workflow for ML projects, and share project materials with a Git repository URL. "Makefiles for data and ML projects" - DVC pipelines are makefiles for ML, with optimizations and human-readable formatting ideal for organizing ML projects. Pipelines connect scripts and dependencies, like datasets and models, for reproducibility and efficiency. “Experiment tracking via Git” - Compare model metrics, hyperparameters, and plots across commits, branches, and releases.. Short History:
DVC was created in 2017 to address gaps in ML tools, and has evolved into a successful open source project with 150+ contributors and thousands of users.
Some interesting highlights from the community:
PythonBytes podcast with makers of spaCy, an NLP library, on integrating with DVC Martin Fowler blog: automating the end-to-end lifecycle of ML applications How Qonto Engineering is using DVC
https://morioh.com/p/ac711e5a9834
https://censius.ai/blogs/ci-cd-for-data-science-and-software-development
Please read and watch everything about DVC and CML
i think you will get the point and make some changes on your code. I wish you good luck