Run build job in private subnet

[chrisammon3000]

Description

• Configures the build job to run in a private subnet
• Removes external NAT Gateway validation (since gateways outside the account can't be validated) and associated environment variables
• Removes VPC endpoint configuration and modification using AWS CLI and associated environment variables (now handled using an optional CloudFormation stack)
• Optionally deploys VPC endpoints using CloudFormation and DEPLOY_VPC_ENDPOINTS
• USE_PRIVATE_SUBNET is true by default (will remove support for deploying to public subnet later)
• Environment variable defaults are updated in the root Makefile
• Deploys and runs using both an external VPC (CREATE_VPC=false) or creating its own VPC (CREATE_VPC=true)

Usage

Private deployment using existing VPC, NAT Gateway and VPC Endpoints

Follow these steps to deploy gfe-db to an existing private subnet and VPC.

• Configure AWS Credentials.
• Create and configure the following resources before deploying the stacks:
  • VPC
  • NAT Gateway
  • Private Subnet with a route to the NAT Gateway (instead of an Internet Gateway)
  • VPC Endpoints:
    • com.amazonaws.us-east-1.s3 (associate with Neo4jDatabaseSecurityGroup and BuildServerSecurityGroup)
    • com.amazonaws.us-east-1.ssm (associate with Neo4jDatabaseSecurityGroup and BuildServerSecurityGroup)
    • com.amazonaws.us-east-1.secretsmanager (associate with Neo4jDatabaseSecurityGroup and BuildServerSecurityGroup)
    • com.amazonaws.us-east-1.ec2messages (associate with Neo4jDatabaseSecurityGroup and BuildServerSecurityGroup)
    • com.amazonaws.us-east-1.ssmmessages (associate with Neo4jDatabaseSecurityGroup and BuildServerSecurityGroup)
    • com.amazonaws.us-east-1.logs (associate with BuildServerSecurityGroup)
    • com.amazonaws.us-east-1.ecr.dkr (associate with BuildServerSecurityGroup)
    • com.amazonaws.us-east-1.ecr.api (associate with BuildServerSecurityGroup)
    • com.amazonaws.us-east-1.ecs (associate with BuildServerSecurityGroup)
• Define the environment variables in .env.dev.

  # .env.dev
  AWS_PROFILE=default
  CREATE_VPC=false
  DEPLOY_NAT_GATEWAY=false
  DEPLOY_VPC_ENDPOINTS=false
  DEPLOY_BASTION_SERVER=false
  ADMIN_EMAIL=<email>
  SUBSCRIBE_EMAILS=<email1,email2,email3,...>
  VPC_ID=vpc-xxxxxxxx
  PUBLIC_SUBNET_ID=subnet-xxxxxxxx
  PRIVATE_SUBNET_ID=subnet-xxxxxxxx
  NEO4J_AMI_ID=<ami_id>
  CREATE_NEO4J_USERS=<username1:password,username2:password,...>
  GITHUB_PERSONAL_ACCESS_TOKEN=ghp_xxxxxxxxxxxxxx
  FEATURE_SERVICE_URL=https://feature.b12x.org
  DOCKER_USERNAME=<username>
  DOCKER_PASSWORD=<password>

• For the stage dev, run STAGE=dev make deploy to deploy the architecture.
• After deployment is complete run STAGE=dev make database.load.run releases="3560" to load the database with the IMGT/HLA release version 3560.

Next Steps

• Remove support for deploying Neo4j in a public subnet