search

aws / aws-step-functions-data-science-sdk-python

Step Functions Data Science SDK for building machine learning (ML) workflows and pipelines on AWS
Apache License 2.0
285 stars 87 forks source link

feat: Support placeholders for TransformStep #157

Closed ca-nguyen closed 2 years ago

ca-nguyen commented 3 years ago

Description

Please include a summary of the change being made.

Fixes #117 and #171

Why is the change necessary?

Currently, it is not possible to use placeholders for Sagemaker Transform properties. The properties cannot be defined dynamically, as they need to be defined in the Transformer which does not accept placeholders. This change makes it possible to use placeholders for Transform properties by using the parameters field that are passed down from the TransformStep.

Solution

Use the parameters field that is compatible with placeholders to define TrainingStep properties. Merge the parameters that were generated from the Transformer with the input parameters:

Testing

Added integration and unit tests

Pull Request Checklist

Please check all boxes (including N/A items)

Testing

Documentation

Title and description

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license.

ca-nguyen commented 3 years ago

The tests pass locally - will need more investigation for throttling issues during build

StepFunctions-Bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository