Addition of developer guides and improvement to READMEs
Ability to deploy ML environment across multiple “organisational units” in the form of sets of dev, preprod and prod accounts, via a centralized governance account. Each set of dev, preprod, and prod gets their own governance CICD pipeline (CodePipeline) to update the “base infra” (mlops_infra) and the “sagemaker project template” (mlops_sm_project_templates) in those accounts.
Addition of a “deploy SageMaker Pipeline” (batch inference) template. This has been a recurring ask from users and customers to be able to:
run batch inference in preprod and prod (in the form of SageMaker Pipelines)
but also in general promote SageMaker Pipelines to preprod and prod accounts (eg for processes that run training and inference within the same SageMaker Pipeline, typical for forecasting projects)
Creation of dedicated SSM Parameters (AWS Systems Manager Parameter Store) for each instantiated project. This allows for a smoother experience when instantiating “dynamic accounts” projects where the account IDs of the preprod and prod accounts are passed as SageMaker Project parameters.
Improvements to CICD constructs to allow SM Projects “deploy” CICD pipelines to be triggered either via Model Registry or via the “build” CICD pipeline finishing. (The latter can be useful for use cases that do not necessarily leverage SM Model Registry)
Note for "deploy SageMaker (batch) pipeline" template: We used the approach to promote SageMaker Pipeline definitions “body” as a json file to preprod and prod via the artefacts bucket, store the s3 location in SSM, and use CDK in the deploy repository to generate a SageMaker Pipeline via infrastructure as code using the json “body”. This allows for a clear separation of responsibilities and avoid repeated code between the “build” and “deploy” repositories. Limitation: SageMaker Pipeline does not accept to be passed roles and networking via sagemaker.workflow.parameters (SM pipeline inputs parameters), so the roles and networking passed to the pipeline for preprod and prod need to happen via string replacement in the SM Pipeline’s json body. (workaround)
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Description of changes:
Addition of developer guides and improvement to READMEs
Ability to deploy ML environment across multiple “organisational units” in the form of sets of dev, preprod and prod accounts, via a centralized governance account. Each set of dev, preprod, and prod gets their own governance CICD pipeline (CodePipeline) to update the “base infra” (mlops_infra) and the “sagemaker project template” (mlops_sm_project_templates) in those accounts.
Addition of a “deploy SageMaker Pipeline” (batch inference) template. This has been a recurring ask from users and customers to be able to:
Creation of dedicated SSM Parameters (AWS Systems Manager Parameter Store) for each instantiated project. This allows for a smoother experience when instantiating “dynamic accounts” projects where the account IDs of the preprod and prod accounts are passed as SageMaker Project parameters.
Improvements to CICD constructs to allow SM Projects “deploy” CICD pipelines to be triggered either via Model Registry or via the “build” CICD pipeline finishing. (The latter can be useful for use cases that do not necessarily leverage SM Model Registry)
Note for "deploy SageMaker (batch) pipeline" template: We used the approach to promote SageMaker Pipeline definitions “body” as a json file to preprod and prod via the artefacts bucket, store the s3 location in SSM, and use CDK in the deploy repository to generate a SageMaker Pipeline via infrastructure as code using the json “body”. This allows for a clear separation of responsibilities and avoid repeated code between the “build” and “deploy” repositories. Limitation: SageMaker Pipeline does not accept to be passed roles and networking via sagemaker.workflow.parameters (SM pipeline inputs parameters), so the roles and networking passed to the pipeline for preprod and prod need to happen via string replacement in the SM Pipeline’s json body. (workaround)
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.