Contents:
00:00 Welcome
00:20 Why Hitchhiker's Guide to MLOps
01:20 Davide Introduces Himself
02:14 Do I need to care about MLOps?
03:40 KPI for ML Models? and do I need to monitor my ML?
06:09 Talk outline
06:30 DevOps Outline
07:30 Traditional SW vs ML
08:33 What is MLOps
09:10 Diagram of MLOps Processes
11:10 What Happens After Deployment (data changes)
12:08 Tracking with MLflow, what is MLflow
13:27 Component and Features of MLflow
14:10 MLflow Experiments and Runs
15:24 How to use MLflow in code (experiment)
16:08 What should we track with MLflow?
17:01 Danger of Data Drift
18:05 How to use Mlflow in code (logging metrics)
19:40 How to use Mlflow in code (context manager)
20:30 How to use Mlflow in code (logging tags)
20:55 How to use Mlflow in code (logging dataset or configuration as artifacts)
21:55 Tracking Model Quality as metrics of an MLflow runs
22:55 MLflow autologging with common ML libraries
23:40 How to use Mlflow in code (autologging with scikit-learn)
24:30 Monitoring with Grafana
26:05 Example of Grafana Dashboard for MLFlow runs
26:55 Setting alerts on Grafana
27:45 Connecting to metrics using REST API
28:50 Wrapping Up: Do I need to monitor ML?
30:25 Wrapping Up: Notes for ML Product Owner
31:35 More Resources
32:21 Q&A
32:50 Q1: What's your opinion on MLflow model serving?
34:32 Q2: What's your recommendation on processing logs?
35:55 Q3: Which MLOps Community You could be part of for starting out?
36:55 Q4: Are you also using MLFlow for monitoring input and output of model?
38:10 Q5: When do you think we should start hiring MLOps Engineer?
Video link: https://youtu.be/_M0sLnZTLog
Video Title: Davide Frazzetto - A Hitchhiker Guide to MLOps | PyData London 2022
Resources: Abstract: https://london2022.pydata.org/cfp/talk/3U3RJZ/ Repository for Presentation: https://github.com/dave-frazzetto/hitchhiker-guide-mlops
Contents: 00:00 Welcome 00:20 Why Hitchhiker's Guide to MLOps 01:20 Davide Introduces Himself 02:14 Do I need to care about MLOps? 03:40 KPI for ML Models? and do I need to monitor my ML? 06:09 Talk outline 06:30 DevOps Outline 07:30 Traditional SW vs ML 08:33 What is MLOps 09:10 Diagram of MLOps Processes 11:10 What Happens After Deployment (data changes) 12:08 Tracking with MLflow, what is MLflow 13:27 Component and Features of MLflow 14:10 MLflow Experiments and Runs 15:24 How to use MLflow in code (experiment) 16:08 What should we track with MLflow? 17:01 Danger of Data Drift 18:05 How to use Mlflow in code (logging metrics) 19:40 How to use Mlflow in code (context manager) 20:30 How to use Mlflow in code (logging tags) 20:55 How to use Mlflow in code (logging dataset or configuration as artifacts) 21:55 Tracking Model Quality as metrics of an MLflow runs 22:55 MLflow autologging with common ML libraries 23:40 How to use Mlflow in code (autologging with scikit-learn) 24:30 Monitoring with Grafana 26:05 Example of Grafana Dashboard for MLFlow runs 26:55 Setting alerts on Grafana 27:45 Connecting to metrics using REST API 28:50 Wrapping Up: Do I need to monitor ML? 30:25 Wrapping Up: Notes for ML Product Owner 31:35 More Resources 32:21 Q&A 32:50 Q1: What's your opinion on MLflow model serving? 34:32 Q2: What's your recommendation on processing logs? 35:55 Q3: Which MLOps Community You could be part of for starting out? 36:55 Q4: Are you also using MLFlow for monitoring input and output of model? 38:10 Q5: When do you think we should start hiring MLOps Engineer?