Closed stefannica closed 3 months ago
[!IMPORTANT]
Review skipped
Auto reviews are disabled on this repository.
Please check the settings in the CodeRabbit UI or the
.coderabbit.yaml
file in this repository. To trigger a single review, invoke the@coderabbitai review
command.You can disable this status message by setting the
reviews.review_status
tofalse
in the CodeRabbit configuration file.
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?
@schustmi I might need you to take a quick look at the side-change documented in the PR description. It affects the way we build docker container images to run pipelines.
LLM Finetuning template updates in examples/llm_finetuning
have been pushed.
LLM Finetuning template updates in examples/llm_finetuning
have been pushed.
Describe changes
Implements a new automated experience of deploying cloud infrastructure and registering full cloud ZenML stacks two-in-one with minimal user input. This experience is meant to reduce the friction and technical difficulties usually encountered when a user has to create a cloud ZenML stack from scratch.
The
zenml stack deploy
CLI command has been repurposed to support this new experience as showcased here:AWS is currently the only supported provider with Google and Azure to follow shortly.
Side changes
Make the
zenml
repository name configurable in the container registry stack componentCurrently, ZenML uses the
.../zenml
docker repository by default for all pipeline images it builds/pushes. This can be overridden, but it's a docker settings that must be configured in code. At the same time, we want these full stack deployment templates to create unique cloud resources, which includes container registries. Therefore, we need the ability to configure the container registry stack components to use a "default repository name" that is different thanzenml
. This PR also adds that capability by modeling a new (optional)default_repository
configuration attribute to all container registry stack components.Use the region from the AWS service connector in the S3 artifact store
The S3 artifact store doesn't explicitly configure the AWS region in the s3fs client. This can lead to errors in environments such as Sagemaker, because the region might default to something else. This PR now uses the AWS region in the service connector to explicitly configure the s3fs client.
Deprecations
The
zenml stack deploy
CLI command has been renamed tozenml stack deploy-mlstack
and marked as deprecated. Users should use the newzenml stack deploy
CLI command to benefit from the new experience or use the mlstacks project stacks separately.Pre-requisites
Please ensure you have done the following:
develop
and the open PR is targetingdevelop
. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.Types of changes