(aws-stepfunctions-tasks): sagemaker runtime invoke endpoint

[earljespino]

Describe the feature

We need the ability to trigger a Sagemaker Endpoint from a step function. I am seeing that there are some available Sagemaker tasks here, but I am not seeing one for Sagemaker Runtime Invoke Endpoint.

In the AWS console's Step Function Workflow Studio, but we are not seeing this as an option in the CDK

Use Case

Need to trigger sagemaker endpoint from step function using CDK

Proposed Solution

No response

Other Information

No response

Acknowledgements

• [X] I may be able to implement this feature request
• [ ] This feature might incur a breaking change

CDK version used

2.37.1 (build f15dee0)

Environment details (OS name and version, etc.)

macOS Monterey 12.4

[earljespino]

Just an update, I was able to implement an 'Invoke Sagemaker Runtime Endpoint' state with a custom state using CustomState.

[kaizencc]

That's exactly what we want to see! That's what CustomState is for. This request will be marked p2 until we have a strong signal from the community that native support is necessary.

[GilbertoCunha]

@earljespino I am trying to achieve the same task (ie, invoking a sagemaker endpoint from inside stepfunctions), but am not sure about what the input to sagemaker should look like.

I created my invoke endpoint task using the following json:

{
    "Type": "Task",
    "End": True,
    "Parameters": {
        "Body": "$.Payload",
        "EndpointName": ENDPOINT_NAME
    },
    "Resource": "arn:aws:states:::aws-sdk:sagemakerruntime:invokeEndpoint",
    "ResultPath": "$.InvocationResult"
}

However, I am having difficulty passing the correct data to the endpoint. I assumed all I had to give it would be the data it needs to output the predictions (passed via the $.Payload argument in the previous task), but this does not work.

I host my endpoint inside a docker image running FastAPI (responding to the /invocations method), and FastAPI throws an error that the input it has received is in an incorrect format. Did you have to change the contents you send to the invoke sagemaker endpoint task? My guess would be passing it the json for a full HTTP request (instead of just the body).

Thanks in advance

[GilbertoCunha]

Either way, for all it's worth, I do believe this use case merits its own task in the CDK.

[Ceaustin117]

Either way, for all it's worth, I do believe this use case merits its own task in the CDK.

Hi, Thanks for the update. Were you able to get something working? What payload did it need?

[GilbertoCunha]

Either way, for all it's worth, I do believe this use case merits its own task in the CDK.

Hi, Thanks for the update. Were you able to get something working? What payload did it need?

It turns out it had nothing to do with the Payload itself. What you send in that Body is exactly what you'll receive in the body of your "/invocations" route for the sagemaker model. However, whenever you want to pass an input/output of another state using Stepfunctions state machine language, you have to add an ".$" at the end of that key's name. Therefore, you just need to change "Body" to "Body.$" in my previous comment:

{
    "Type": "Task",
    "End": True,
    "Parameters": {
        "Body.$": "$.Payload",
        "EndpointName": ENDPOINT_NAME
    },
    "Resource": "arn:aws:states:::aws-sdk:sagemakerruntime:invokeEndpoint",
    "ResultPath": "$.InvocationResult"
}

[joekendal]

+1