awslabs / data-on-eks

DoEKS is a tool to build, deploy and scale Data & ML Platforms on Amazon EKS
https://awslabs.github.io/data-on-eks/
Apache License 2.0
550 stars 180 forks source link

Support S3 gateway endpoint for EMR on EKS #553

Open hitsub2 opened 3 weeks ago

hitsub2 commented 3 weeks ago

Community Note

What is the outcome that you are trying to reach?

In data scenarios, accessing s3 would have a high cost for network traffic.By gateway endpoint for s3, we can reduce the cost which will benefit the customer.

Describe the solution you would like

Implementing s3 gateway endpoint for emr on eks.

Describe alternatives you have considered

Additional context

vara-bonthu commented 2 weeks ago

The S3 Gateway endpoint is included in the blueprint. Check the code here: vpc.tf#L93.

By default, it is disabled. To enable the endpoints, update the configuration from this link: variables.tf#L40.