Closed strickvl closed 10 months ago
Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
Our GitHub checks need improvements? Share your feedbacks!
This PR builds on the work listed in the PRD to improve the
mlops-stack
UX.TL;DR:
mlstacks
Python package that people can use to deploy MLOps stacks.mlstacks
(see this sister PR for the details) for deployment of these stacks / stack components.tf
file changes. These have mainly been copy-pasted + duplicated for backwards compatibility. The core changes are docs + Python code in this PR.‼ Key Changes / Updates
mlstacks
is a Python package now (different from how we previously did things)..yaml
) files in order to use themlstacks
package / logic.⦾ Core concepts / mental models
The core flow is as follows:
The stack and component specs are defined through Pydantic models (defined in
src/mlstacks/models/
directory) which handles the high-level validation.The Terraform files / modules themselves are (currently) located at
src/mlstacks/terraform
, though our current idea is to also move those out to the Terraform Registry as soon as is possible.NOTE: there are a bunch of terraform modules / directories located at the root of the
mlops-stacks
repository. Those are there to handle backwards compatibility (i.e. people using old versions of ZenML, since we don't want to prevent them from using stack recipes the old way if they want).Video Walkthrough
Loom Video Link
👷 How to try me out
What follows are some simple CLI commands you can use to try things out to get a sense of how it all works. Within
mlstacks
we only allow for the deployment of stacks (i.e. deployment of a single component requires you to define it in relation to a stack).First things first: you'll need to install the package into your local environment:
🥞 Stack Deployment Example Use
First create two files.
simple_stack.yaml
first:Then
simple_component_gcs.yaml
:To deploy an artifact store to GCP:
Then you can check on GCP that the bucket exists here.
Then you can delete the artifact-store / bucket with the following command:
= Deployment Outputs Example Use
(You'll need to have something deployed in order for this to work.)
To view the outputs for a particular deployment:
This will print out a list of the values. You can also get a specific k:v pair in the following way:
💰 Infracost Cost Estimation Example Use
To get a cost estimation for a particular stack:
🚧 Remaining / Missing Tasks
(For the full breakdown of remaining tasks, see our board on Notion.)