Closed victorgu-github closed 1 year ago
Based on the comments from Vara and Apoorva, replace ACK and just use terraform
Lets push this feature to EMR on EKS Workshop content
This issue has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this issue will be closed in 10 days
Issue closed due to inactivity.
Community Note
What is the outcome that you are trying to reach?
[AWS Step Functions] which is a serverless workflow service can integrate with Amazon EMR on EKS and [Amazon EventBridge] to build event-driven workflows. After installing [AWS Controllers for Kubernetes (ACK) in EKS, you can provision and configure serverless AWS resources: Amazon EventBridge and AWS Step Functions from EKS. The team can do the whole data operation without leaving the Kubernetes platform and only need to maintain the EKS cluster since all the other components are serverless.
The airflow infra is too heavy for some users. An data pipeline with serverless services will offload a lot from admin works while ACK controllers allow them to stay in EKS to control and config those serverless services.
Describe the solution you would like
Describe alternatives you have considered
airflow, argo workflows
Additional context