search

aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.7k stars 3.93k forks source link

aws-cdk-lib.aws_stepfunctions_tasks: Support optimized integration with EMR Serverless #32061

Open qhaoran opened 2 weeks ago

qhaoran commented 2 weeks ago

Describe the feature

Hi CDK team,

Step function already supported the optimized integration of EMR serverless since late 2023 (see ref). And we can integrate the EMR serverless this way on AWS console when manually creating the step function. However, the CDK library doesn't have such interface in aws-cdk-lib.aws_stepfunctions_tasks module. This forces us to still call the AWS service API to be able to integrate to step function.

Use Case

We have lots of use cases that orchestrate the EMR serverless job with step function, and we love managing the infra in CDK. The lack of this interface prevents us leveraging the optimized integration approach. Currently we have to build another separate step function to call EMR serverless API in order to submit Spark job, poll the status and return result. And all these CDK code, step function are no longer needed if CDK library can support the optimized integration with EMR serverless.

And there're people looking for the same (see discussion)

Proposed Solution

No response

Other Information

No response

Acknowledgements

CDK version used

CDK v2 2.166.0

Environment details (OS name and version, etc.)

Linux

khushail commented 1 week ago

Hi @qhaoran , Yes, EMR Serverless intergationg with stepfunction tasks would surely would be helpful to have. Appreciate you volunteering for this contribution. Thanks.