search

aws / containers-roadmap

This is the public roadmap for AWS container services (ECS, ECR, Fargate, and EKS).
https://aws.amazon.com/about-aws/whats-new/containers/
Other
5.22k stars 321 forks source link

[ECS] [request]: Alternative to expensive NatGateway in private subnet #419

Open ronkorving opened 5 years ago

ronkorving commented 5 years ago

Tell us about your request

Whenever you need a container to access the internet from a private subnet, you need a NatGateway. PrivateLink does a decent job of removing the need to do so in some cases (like ECR access), but in a connected world, the need for internet access remains high. A NatGateway, especially when using more than 1 AZ, is really quite expensive however. It also consumes an ElasticIP, even when I don't care for a static IP, which causes me to hit service limits. This must be such a common pattern, that I'm wondering if an optimized and cheaper alternative to NAT Gateway for ECS private subnets cannot be found.

Which service(s) is this request for?

Fargate, ECS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

Not hard, just a big portion of the monthly bill, for such limited business value.

Are you currently working around this issue?

No

Additional context

Attachments

pauldraper commented 5 years ago

On Fargate you can just have a public IP address right?

ronkorving commented 5 years ago

@pauldraper Ah good point, I should've been clearer. I want to keep my containers in a private subnet. Updated my original issue accordingly.

whereisaaron commented 5 years ago

@ronkorving yeah NAT Gateway are very expensive, even for modest amounts of traffic, and become a huge overhead on many stacks. AWS makes public IPv4’s cheap, and NAT expensive, which seems backwards.

AWS does still provides the old NAT instance images (or you can build your own). Even a tiny NAT instance can handle a lot of common traffic loads. Deploying a NAT instance per AZ is fairly inexpensive (though upgrades are your problem). I did a rough costing below.

https://github.com/weaveworks/eksctl/issues/392#issue-395816205

Per-AZ AWS NAT Gateways are expensive, especially given the additional traffic charges, (e.g. 3 x AZ and 2TB traffic = $250/month in Sydney), versus per-AZ t3.micro NAT instances (e.g. 3 x t3.micro and 2TB traffic = $15/month in Sydney). So NAT gateways are ~15 times more expensive to run for small amounts of (mostly image pull) traffic. And it gets worse if you use more traffic.

ronkorving commented 5 years ago

@whereisaaron Thanks for sharing that information. NAT instances are not an option for me, for compliance reasons. In my case, we will always choose the managed option, knowing full well it comes at a premium. And paying a premium for managed resources is fine for us, and to be expected. But as you seem to agree, it's kind of a high premium in the case of NAT Gateways :)

smileatom commented 4 years ago

Im experiencing this now. This is quite a chunk to pay for network traffic. Especially ECR pulls. I may switch to NAT instances even though I would rather not. Not sure if I can at least use PrivateLink for ECR access and at least only pay for ACTUAL traffic through the managed gateways. Either way this is pretty ridiculous cost just to have private redundant subnets.