search

aws-quickstart / quickstart-aws-vpc

Amazon Virtual Private Cloud—AWS Solution
Apache License 2.0
325 stars 283 forks source link

Enhancement: Allow selecting Single AZ deployment to facilitate spinning up low cost non-production environment for quick PoC #65

Open maulik-modi22 opened 1 year ago

maulik-modi22 commented 1 year ago

To have tight control over costs, we would like to use Single AZ deployment to facilitate spinning up low cost non-production environment for quick PoC. Number of availability zones needs an additional option - "1" and related validations if any

single-AZ
tlindsay42 commented 1 year ago

@maulik-modi22 PoC/workshop/non-production use cases are starting to become more common, but production use cases are what we focus on. We don't have any plans to build this feature, but we might be willing to accept a pull request if you want to build it. Are you interested? If so, don't start building it yet as I need to discuss with my team.

maulik-modi22 commented 1 year ago

@tlindsay42 , Single AZ based VPC are need of the hour for people running GPU based workloads for load testing, internal demonstrations, bechmarking as part of CI/CD Pipelines that require automation to spin up VPC, Deploy AWS resources, benchmark ML model training duration performance against given GPU based instance and delete nested stack... Some of the softwares still do not support Graviton based GPU

Sharing an example calculator with just 1 GPU instance of size p3.16xlarge or p4d.24xlarge https://calculator.aws/#/estimate?id=e0fea89c09a8d698ba4991b868c430e0ee31cca9

tlindsay42 commented 1 year ago

@maulik-modi22 Thanks for the additional context about your use case.

tlindsay42 commented 1 year ago

Still debating with my team whether we would be willing to support this feature.

In the meantime, it's possible with existing features to deploy a 2 AZ VPC with the CreateNATGateways parameter disabled (which is the only billable resource deployed by default) and then deploy your instances in the single AZ you'd use, along with a NAT Gateway if needed. Would that work?

maulik-modi22 commented 1 year ago

Let me provide two more use cases where we tend to use single compute node:

  1. https://next.redhat.com/project/microshift/
  2. https://docs.openshift.com/container-platform/4.12/installing/installing_sno/install-sno-preparing-to-install-sno.html
tlindsay42 commented 1 year ago

You're not addressing or answering the questions I'm asking. ?

maulik-modi22 commented 1 year ago

I am looking for pure Single AZ deployment, it seems you are suggesting to deploy multi AZ; here's the side effects:

  1. it would enable spewing of EC2 compute, EBS storage, EFS Access points in any of the multi-AZ.
  2. deployment architecture diagram would show 2 AZ with no compute, EBS or EFS in other AZ - folks would question why is that
  3. how would AWS resource explorer would look like in such deployment
tlindsay42 commented 1 year ago

I am looking for pure Single AZ deployment, it seems you are suggesting to deploy multi AZ

Deploying an unused subnet in a second AZ doesn't make an architecture multi-AZ, same as deploying in a region with more than one AZ doesn't it multi-AZ just because it's available. How the application is designed on top of that infrastructure is what's important.

  1. it would enable spewing of EC2 compute, EBS storage, EFS Access points in any of the multi-AZ.

Please elaborate. This isn't generally how infrastructure as code solutions work. ?

  1. deployment architecture diagram would show 2 AZ with no compute, EBS or EFS in other AZ - folks would question why is that

Would either adding a design decision note or just not show it in the diagram (since it isn't used) work for you. ?

  1. how would AWS resource explorer would look like in such deployment

Please elaborate. I don't understand what challenge this poses for you and your customers. ?

Last but not least, if my team decides to support a single-AZ VPC feature (which comes with risks that we're not sure we want to accept), would you be interested in building it? If not, this isn't on our roadmap, so we may not get around to it anytime soon. If my team decides that we don't want to support this feature, one of the best things about open source projects like this is that you can fork the project and customize as needed for your use case.

In case others stumble across this and want this feature, please add a 👍 to the original comment to help us gauge interest.

maulik-modi22 commented 1 year ago

What if you keep a separate VPC template altogether?

tlindsay42 commented 1 year ago

We have no interest in that for now, but you're welcome to start your own fork.

maulik-modi22 commented 1 year ago

I am able to create VPC using AWS Console in 1 AZ but this cannot be automated.

How do I achieve this using your VPC template- 1 Internet gateway AZ1 - 1 Private subnet, 1 Public subnet with 1 NATGateway AZ2 - Just subnets

deploy a 2 AZ VPC with the [CreateNATGateways parameter](https://github.com/aws-quickstart/quickstart-aws-vpc/blob/9dc47510f71f1fb6baf8c4e96b5330a6f51f540e/templates/aws-vpc.template.yaml#L148-L156) disabled (which is the only billable resource deployed by default) and then deploy your instances in the single AZ you'd use, along with a NAT Gateway if needed.

tlindsay42 commented 1 year ago

It can be automated. One way to go about it is to build a new CloudFormation template that deploys the VPC template, kinda like this, but with the parameters configured the way that you want, like CreateNATGateways set to 'false', and then add the additional resources needed, like the NAT gateway, using the VPC template as a guide. Here's the CloudFormation Resource Type Reference docs for EC2 (and VPC) too.