hackoregon / civic-devops

Master collection point for issues, procedures, and code to manage the HackOregon Civic platform
MIT License
11 stars 4 forks source link

Update master.yaml in hackoregon-aws-infrastructure to be current with production cluster #94

Closed MikeTheCanuck closed 6 years ago

MikeTheCanuck commented 6 years ago

The hacko-integration ECS cluster is likely running with different configuration than what you'll find in our two repositories:

  1. GitHub
  2. S3

As step 1 towards working on #91, we need to figure out what is the current configuration of the cluster.

Presumably we need an up-to-date master.yaml to capture that state.

Gotta figure out which of the two repos has the more-current settings, and then figure out a way to compare what's live in the hacko-integration cluster with the more up-to-date config from the repos.

MikeTheCanuck commented 6 years ago

So I've got current copies of the two repos' master.yaml and in the preamble is a pretty clear clue:

  1. GitHub: "Last Modified: 17th February 2017"
  2. S3: "Last Modified: 13th May 2017"
MikeTheCanuck commented 6 years ago

There is exactly one diff between the two:

GitHub

Resources:
    transportService:
        Properties:
            Parameters:
                ConfigBucket: hacko-transport-config

S3

Resources:
    transportService:
        Properties:
            Parameters:
                ConfigBucket: hacko-transportation-config
MikeTheCanuck commented 6 years ago

I've updated the GitHub copy here with the later enhancements found in S3: https://github.com/hackoregon/hackoregon-aws-infrastructure/pull/3

Now we just have to determine if there were any other changes to the hacko-integration cluster that don't appear in either repo's master.yaml.

MikeTheCanuck commented 6 years ago

Production configuration

I have discovered that the following command returns the current "template" for a specified "stack":

aws cloudformation get-template --stack-name hacko-integration > master.yaml

It appears that this contains the code from master.yaml for the current cluster hacko-integration. However, it returns it mangled with \n line feed characters, so that it's impossible to generate an automated comparison between the readable YAML and the result of get-template - this is a snippet of the first few "lines" of the response from get-template:

{
    "TemplateBody": "Description: >\n\n    This template deploys a VPC, with a pair of public and private subnets spread\n    across two Availabilty Zones. It deploys an Internet Gateway, with a default\n    route on the public subnets. It deploys a pair of NAT Gateways (one in each AZ),\n    and default routes for them in the private subnets.\n\n    It then deploys a highly available ECS cluster using an AutoScaling Group, with\n    ECS hosts distributed across multiple Availability Zones.\n\n    It then deploys a Highly available RDS Multi Availability Zone database services service using ()\n\n    Finally, it deploys a pair of example ECS services from containers published in\n    Amazon EC2 Container Registry (Amazon ECR).\n\n    Based on AWSLabs ECS Reference Arhitecture\n    https://github.com/awslabs/ecs-refarch-cloudformation\n\n    Last Modified: 13th May 2017\n    Author Dan Carr (ddcarr@gmail.com)\n\n
iant01 commented 6 years ago

As I indicated yesterday I have a means to get the templates that were last used last year in May and will be getting them tomorrow so they can be compared against what is in S3.

iant01 commented 6 years ago

obtained copies of templates used for last CF update of the current Hacko AWS environment (excluding the database servers). Compared all master.yaml, vpc.yaml, security-groups.yaml, load-balancers.yaml, ecs-cluster.yaml, ec2-instance.yaml (used for bastion host) and the service.yaml files in each of budget, civic-lab, emergency, endpoints, homeless, housing and transport against the CF templates in S3 /hacko-infrastructure-cfn.

These are the difference between all the templates: master.yaml ecs instance type is t2.large in current stack and t2.micro in S3 version and github version In transport service ProjSettingDir value is transDjango in current stack and transportationAPI in S3
version and github version

civic-lab-service/service.yaml memory size value is 1GB (1024) in current stack and 100meg in S3 version and github version

Will create a separate issue for civic-lab-service item to track separately.