[Enhancement]: Option for Provisioned Concurrency in server-less SageMaker

[jkubajek]

Description

Use Case

I would like to set-up the SageMaker server-less inference instance and I would like to specify the ProvisionedConcurrency attribute. Here you can find an official blog post from AWS on how to set-up ProvisionedConcurrency manually - https://aws.amazon.com/blogs/machine-learning/announcing-provisioned-concurrency-for-amazon-sagemaker-serverless-inference/

Current situation

Currently, Terraform in the serverless_config allows to specify only max_concurrency and memory_size_in_mb. https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sagemaker_endpoint_configuration#serverless_config

Proposal

I would like to have additional optional field in the serverless_config called provisioned_concurrency which would allow me to specify the ProvisionedConcurrency in the configuration of SageMaker.

Proposed changes

Below I put the minimal modification of the terraform-provider-aws/internal/service/sagemaker /endpoint_configuration.go. I am only not sure how to set default value of provisioned_concurrency to 0, so that current config would provide the same output as so far. There should be also a verification on whether the provisioned_concurrency is not greater that max_concurrency.

Lines 325 and 442

"serverless_config": {
    Type:     schema.TypeList,
    Optional: true,
    MaxItems: 1,
    ForceNew: true,
    Elem: &schema.Resource{
        Schema: map[string]*schema.Schema{
            "max_concurrency": {
                Type:         schema.TypeInt,
                Required:     true,
                ForceNew:     true,
                ValidateFunc: validation.IntBetween(1, 200),
            },
            "memory_size_in_mb": {
                Type:         schema.TypeInt,
                Required:     true,
                ForceNew:     true,
                ValidateFunc: validation.IntInSlice([]int{1024, 2048, 3072, 4096, 5120, 6144}),
            },
            "provisioned_concurrency": {
                Type:         schema.TypeInt,
                Required:     false,
                ForceNew:     true,
                ValidateFunc: validation.IntBetween(0, 200),
            },
        },
    },
},

Line 904

func expandServerlessConfig(configured []interface{}) *sagemaker.ProductionVariantServerlessConfig {
    if len(configured) == 0 {
        return nil
    }

    m := configured[0].(map[string]interface{})

    c := &sagemaker.ProductionVariantServerlessConfig{}

    if v, ok := m["max_concurrency"].(int); ok {
        c.MaxConcurrency = aws.Int64(int64(v))
    }

    if v, ok := m["memory_size_in_mb"].(int); ok {
        c.MemorySizeInMB = aws.Int64(int64(v))
    }

    if v, ok := m["provisioned_concurrency"].(int); ok {
        c.ProvisionedConcurrency = aws.Int64(int64(v))
    }

    return c
}

Line 1022

func flattenServerlessConfig(config *sagemaker.ProductionVariantServerlessConfig) []map[string]interface{} {
    if config == nil {
        return []map[string]interface{}{}
    }

    cfg := map[string]interface{}{}

    if config.MaxConcurrency != nil {
        cfg["max_concurrency"] = aws.Int64Value(config.MaxConcurrency)
    }

    if config.MemorySizeInMB != nil {
        cfg["memory_size_in_mb"] = aws.Int64Value(config.MemorySizeInMB)
    }

    if config.ProvisionedConcurrency != nil {
        cfg["provisioned_concurrency"] = aws.Int64Value(config.ProvisionedConcurrency)
    }

    return []map[string]interface{}{cfg}
}

Affected Resource(s) and/or Data Source(s)

No response

Potential Terraform Configuration

serverless_config {
      max_concurrency = 10
      memory_size_in_mb = 4096
      provisioned_concurrency = 1
}

References

https://aws.amazon.com/blogs/machine-learning/announcing-provisioned-concurrency-for-amazon-sagemaker-serverless-inference/

Would you like to implement a fix?

None

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[DrFaust92]

Hi, there is an open PR to support this https://github.com/hashicorp/terraform-provider-aws/pull/31398

[github-actions[bot]]

This functionality has been released in v5.1.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.