Competition

[MikoMikarro]

Tracking and Metadata Management

MLflow

• Open Source
• MLflow Tracking: storing and accessing code, data, configuration, and results.
• MLflow Projects: package data science source for reproducibility.
• MLflow Models: deploying and managing machine learning models to various serving environments. 
• MLflow Model Registry: a central model store that provides versioning, stage transitions, annotations, and managing machine learning models.

  Comet ML

• Tracking, comparing, explaining, and optimizing machine learning models and experiments. You can use it with any machine learning library, such as Scikit-learn, Pytorch, TensorFlow, and HuggingFace.

  Weights & Biases

• Experiment tracking, data and model versioning, hyperparameter optimization, and model management. Furthermore, you can use it to log artifacts (datasets, models, dependencies, pipelines, and results) and visualize the datasets (audio, visual, text, and tabular).

  ZenML

Orchestration and MLOps

Prefect

• Open Source
• Monitoring, coordinating, and orchestrating workflows between and across applications

  Kedro

• Creating reproducible, maintainable, and modular data science projects. It integrates the concepts from software engineering into machine learning, such as modularity, separation of concerns, and versioning.

  [[#[Kubeflow](https //www.kubeflow.org/docs/)|Kubeflow]]

  [[#[Data Version Control](https //dvc.org/)|DVC]]

  Data and Pipeline Versioning Tools

  Pachyderm

• Automates data transformation with data versioning, lineage, and end-to-end pipelines on Kubernetes.
• The community edition is open-source.
• "GIT for ML"

  Data Version Control

• Open Source
• It works seamlessly with Git to provide you with code, data, model, metadata, and pipeline versioning.
• Extra features
  • Experiment tracking (model metrics, parameters, versioning).
  • Create, visualize, and run machine learning pipelines. 
  • Workflow for deployment and collaboration.
  • Reproducibility.
  • Data and model registry.
  • Continuous integration and deployment

    Sacred

• Open Source
• Configure, organize, log and reproduce computational experiments. It is designed to introduce only minimal overhead while encouraging modularity and configurability of experiments.
• Features
  • keep track of all the parameters of your experiment
  • easily run your experiment for different settings
  • save configurations for individual runs in files or a database
  • reproduce your results

    Model Monitoring

    Evidently AI

• Open Source
• Python
• Checks data and model quality, data drift, target drift, and regression and classification performance.

  End to end

  Kubeflow

• Makes machine learning model deployment on Kubernetes simple, portable, and scalable.
• Features
  • Centralized dashboard with interactive UI. 
  • Machine learning pipelines for reproducibility and streamlining. 
  • Hyperparameter tuning and neural architecture search.
  • Training jobs and Job scheduling. 
  • Works with all of the major cloud providers.

    References

• https://www.datacamp.com/blog/top-mlops-tools
• https://neptune.ai/blog/mlops-tools-platforms-landscape
• https://ml-ops.org/content/state-of-mlops
• https://medium.com/@lktsdvd/navigating-the-mlops-landscape-an-in-depth-look-at-essential-mlops-tools-30ea889b2a2
• From DevOps to MLOps: Overview and Application to Electricity Market Forecasting
• State of the Art Practices for Machine Learning Operations in Google Cloud
• MLOps Spanning Whole MachineLearningLife Cycle: ASurvey
• MLops papers repository
• Landscape of AI