Timecodes for "MLOps Zoomcamp 6b.3 - Terraform: Build an e2e workflow for ride predictions"

[alexeygrigorev]

Youtube video: https://www.youtube.com/watch?v=JVydd1K6R7M

[dimzachar]

0:00:03 - Configure pipeline end-to-end with example. 0:00:51 - Creating two Kinesis streams. 0:02:14 - Create S3 bucket for model artifacts. 0:03:29 - Create bucket with Terraform, set ACL to private. 0:04:30 - Create private AWS bucket, output variable. 0:05:25 - Create and configure resources with Terraform. 0:06:12 - Creating S3 bucket for project use. 0:07:56 - Reference model bucket and project ID in terraform. 0:09:07 - Create S3 bucket with ${SDG} input. 0:10:16 - Create custom naming for ML flow. 0:11:05 - Creating a new bucket for model storage. 0:12:05 - Ignore warnings, configure 3 resources. 0:12:49 - Configuring ECI registry & Lambda. 0:14:35 - Creating resources in parallel for Lambda dependencies. 0:15:16 - Model fails without source, ECR, Kinesis, DST. 0:15:57 - Configure ECR repo, set image tag mutability. 0:17:11 - Configure docker image for Lambda. 0:18:08 - Managing infrastructure code with CI/CD. 0:19:02 - Create dependency on ECR for Lambda. 0:19:49 - Create ECR image, upload via CI/CD. 0:20:40 - Package Lambda to ECR registry. 0:21:38 - Create ECR repo, fetch URL, input image tag. 0:22:31 - Create variables, pass data source, execute bash command. 0:23:38 - Configuring ECR image for tracking changes. 0:24:30 - Creating an ECR image with Lambda and outputting the URI. 0:25:37 - Configure ECR module with variables.tf. 0:26:48 - Introducing tfvars for runtime variables. 0:27:53 - Create prod & staging env vars. 0:28:49 - Create environment variables for runtime. 0:30:18 - Set environment variables for staging right predictions ML flow model. 0:31:22 - Navigate to code dir, locate timer func file. 0:32:21 - Create environment with variable file and run. 0:33:57 - Creating ECR resources, login succeeded. 0:35:17 - Revising CI/CD pipeline config. 0:36:13 - Keep a minimalistic docker image in infrastructure repo for consistent URI with latest tag. 0:37:04 - Create Lambda function for resource. 0:38:30 - Configuring environment variables for package, image URI, and rule. 0:39:15 - Passing static values, updating config, assigning timeout, processing docker image. 0:40:05 - Configuring IM rule dependencies. 0:40:57 - Creating IAM role for resources. 0:42:05 - Labeling project ID for Lambda permissions. 0:43:11 - Allow Kinesis processing & role policy attachment. 0:44:15 - Attach policy to role: 3-step format. 0:45:18 - Put records in output Kinesis stream. 0:46:06 - Generate ARN for resource access. 0:47:17 - Manage dependencies for Kinesis module. 0:48:09 - Generate output stream ARN. 0:49:06 - Granting Lambda permissions for CloudWatch logging. 0:50:24 - Granting permissions to model bucket. 0:51:33 - Reference same module for ARN output. 0:52:22 - Creating variables for Lambda. 0:53:23 - Lambda function requires handler and runtime. 0:54:09 - Passing static values for prediction stream and model bucket. 0:54:54 - Model versioning with MLFlow. 0:56:01 - Creating ML flow models bucket. 0:56:56 - Copy artifacts to bucket, execute lambda. 0:57:47 - Creating infrastructure, training models, deploying apps with CI/CD. 0:58:39 - Trigger Lambda on Kinesis stream. 0:59:31 - Create lambda event mapping for Kinesis. 1:00:28 - Trigger event to process Kinesis streams. 1:01:38 - Trigger lambda with CloudWatch. 1:02:33 - Reference lambda function, create dependencies, get ARN of ECR image. 1:03:52 - Pass input, output, source. 1:05:44 - Create lambda function, add resources, allow kinesis access. 1:07:25 - Creating resources with unique names. 1:08:27 - Fixing duplicate resource error. 1:09:16 - Creating lambda function for project and environment. 1:10:26 - Testing policy changes with event.

[amitfrancis]

Updated timecodes. Thank you!