MLOPs/MLFlow on HPG

[YunchaoYang]

Tutorials on using MLflow on HPG

1. Open an OOD terminal

2. Load all required modules

module load conda firefox git sqlite

clone the repo: https://github.com/tsterbak/pydataberlin-2022/blob/main/tutorial/1_Run%20and%20track%20experiments.ipynb

3. create an conda environment with MLflow installed

conda create -n mflow python=3.9
conda activate mlflow
pip install mlflow
pip install scikit-learn
pip install pandas
pip install jupyterlab

4. start thejupyter-lab in the tutorial folder.

jupyter-lab

5. you can open a new terminal from OOD or directly use the terminal in the jupyterlab, type in the following command:

mlflow server --backend-store-uri sqlite:///mflow.db --default-artifact-root mlruns/ --host 0.0.0.0 --port 5000

You may change the port number if it is already in use.

6. open a browser and navigate to the url: 0.0.0.0://5000, now you can see the MLflow GUI interface

7. Continue running the notebook, you can see different runs are updated in the project

8 You may change parameters, and mlflow will create logs for your runs with parameters.

references:

1. https://github.com/tsterbak/pydataberlin-2022
2. https://www.mlflow.org/docs/latest/tutorials-and-examples/tutorial.html

[YunchaoYang]

2. MLFlow Deploy and Manage

1. server the model from the registry

start a new terminal and start the mlflow serve:

MLFLOW_TRACKING_URI=http://localhost:5000 mlflow models serve --no-conda -m "models:/penguins_clf/1" -p 4242

2. query the model using cURL or call the API from python requests

3. Other Deployment targets

• Sagemaker
• AzureML
• Kubernetes

Over the course of the model\u2019s lifecycle, a model evolves\u2014from development to staging to production. You can transition a registered model to one of the stages: Staging, Production or Archived.

[YunchaoYang]

Key concepts in MLFlow

MLflow is an open source platform to manage the ML lifecycle, including experimentation, reproducibility, deployment, and a central model registry. The components of MLFlow includes:

• MLflow Tracking - Record and query experiments: code, data, config, and results
• MLflow Projects - Package data science code in a format to reproduce runs on any platform
• MLflow Models - Deploy machine learning models in diverse serving environments
• Model Registry - Store, annotate, discover, and manage models in a central repository

Install MLflow

pip install mlflow

# Install MLflow with the experimental MLflow Pipelines component
pip install mlflow[pipelines]  # for pip
conda install -c conda-forge mlflow-pipelines  # for conda

# Install MLflow with extra ML libraries and 3rd-party tools
pip install mlflow[extras]

# Install a lightweight version of MLflow
pip install mlflow-skinny

Tutorials and Examples

There is a plethera of examples of MLflow use cases, integrating MLFlow into various projects in Pytorch, Tensorflow1/2, Keras, Scikit-learn, RAPIDS,

Specific purpose examples:

1. Hyperparameter Tuning Example project.
2. Multistep Workflow Example project.

MLflow Models

An MLflow Model is a standard format for packaging machine learning models that can be used in a variety of downstream tools—for example, real-time serving through a REST API or batch inference on Apache Spark.

Model API

model flavor’s mlflow.<model_flavor>.method()

use the mlflow.models.Model class to create and write models

MNIST PyTorch example

• use the [mlflow.pytorch.save_model()](https://www.mlflow.org/docs/latest/python_api/mlflow.pytorch.html#mlflow.pytorch.save_model)and [mlflow.pytorch.log_model()](https://www.mlflow.org/docs/latest/python_api/mlflow.pytorch.html#mlflow.pytorch.log_model)methods to save PyTorch models in MLflow format; both of these functions use the torch.save() method to serialize PyTorch models;

• use the [mlflow.pytorch.load_model()](https://www.mlflow.org/docs/latest/python_api/mlflow.pytorch.html#mlflow.pytorch.load_model) method to load MLflow Models with the pytorch flavor as PyTorch model objects

• mlflow.pytorch.autolog() - Auto log all MLflow entities

import mlflow.pytorch
from mlflow import MlflowClient
...
 # Auto log all MLflow entities
  mlflow.pytorch.autolog()
 # Train the model
  with mlflow.start_run() as run:
      trainer.fit(mnist_model, train_loader)
...
# fetch the auto logged parameters and metrics
print_auto_logged_info(mlflow.get_run(run_id=run.info.run_id))

• mlflow.pytorch.get_default_conda_env()

# Fetch the associated conda environment
env = mlflow.pytorch.get_default_conda_env()
print("conda env: {}".format(env))

Storage Format

• flavors

  my_model/
  ├── MLmodel
  ├── model.pkl
  ├── conda.yaml
  ├── python_env.yaml
  └── requirements.txt

MLmodel Format

MLflow Model Registry

• a centralized model store, set of APIs, and UI, to collaboratively manage the full lifecycle of an MLflow Model.
• It provides model lineage (which MLflow experiment and run produced the model), model versioning, stage transitions (for example from staging to production), and annotations.