Revisit and resolve issues with exisiting VPC support

[vlj91]

From @ste78 on April 27, 2016 8:58

Support an existing VPC in all CF templates rather than need to create a new one for every cluster, document prerequisites on vpc.

Copied from original issue: alces-software/flight-aws-marketplace#23

[vlj91]

From @ste78 on April 27, 2016 9:7

@vlj91 can you link me to your efforts on getting this to work so I can go from there.

[vlj91]

https://raw.githubusercontent.com/alces-software/flight-appliance-support/master/aws-cloudformation/aws-tools/templates/all-in-one/hpc-cluster.json

Some of the bits from here will do what I put in #13 -

In parameters you need:

        "SubnetId": {
            "Description": "Enter the ID of your existing subnet. If you wish to have one created for you, select None. The SecurityGroup field must also be set to None.",
            "Type": "String",
            "Default": "None"
        },
        "SecurityGroup": {
            "Description": "Enter the ID of your existing security group. If you wish to have one created for you, select None. The Subnet ID field must also be set to None.",
            "Type": "String",
            "Default": "None"
        },

The conditions section:

    "Conditions": {
        "CreateNetwork": {
            "Fn::Equals": [
                {
                    "Ref": "SubnetId"
                },
                "None"
            ]
        }
    },

Then as said in #13 I'd suggest having two login node resources, one for if your own network is supplied - one for if a new cluster network is created

[vlj91]

From @ste78 on April 27, 2016 9:18

Thanks, Have we done anything towards the documentation of the pre-requisites on the VPC?

[vlj91]

Only brief notes - can turn into a better detailed guide if needed:

VPC

• DNS hostnames: enabled

Security Group Recommended inbound rules:

SSH enabled (port 22) from your desired IP range Ports 5900-5920 TCP enabled from your desired IP range All traffic enabled between subnet hosts

Recommended outbound rules: All traffic outbound enabled

Internet gateway An internet gateway should be created and attached to the VPC

Route table A route table should be created, linking the subnet to the Internet Gateway

[vlj91]

From @ste78 on May 2, 2016 17:1

Ok so there are a number of problems with this,

1) You can't accept empty string using the AWS::EC2::SecurityGroup::Id or AWS::EC2::Subnet::Id fields so that makes the combination of making something thats acutually useable and can do creating network / using existing network from the same template close to impossible

2) The DependsOn property seems to be evaluated by cloudformations' validators before any of the functions are run, so it tries to validate the "Fn::If" text rather than the actual processed string as a valid resource on which to depend. Googling suggests other people are using multiple templates to get round this issue, which is currently a no-go for marketplace

So conclusion at the moment is that we can't offer one template to do with and without created VPC, so once mix that with the potential cause for problems when people use their own VPCs maybe we need to think of using an existing VPC as a custom non-marketplace option.

[vlj91]

From @mjtko on June 23, 2016 12:56

The thought occurs to me that we could go ahead and create a VPC anyway but not use it. AFAICS, it doesn’t incur any additional cost although it would, of course, not reduce any delay in completing the creation of the CFN stack.

This might be an acceptable compromise though.