clean up and generalize AWS documentation

[danrademacher]

This CloudFormation works to create an EC2 on our AWS account:

aws --profile=gin cloudformation create-stack --stack-name=seattle-energy --template-body file://cloudformation.json --region us-west-1 --parameters ParameterKey=KeyPair,ParameterValue='GreenInfo SSH Key for EC2 US-West-1 Zone' --capabilities CAPABILITY_IAM

using config changes in this commit (on pdf-generator branch): https://github.com/GreenInfo-Network/seattle-building-dashboard/blob/pdf-cloud-formation/pdf-generator/cloudformation.json

Two things I had to do to get the command to (mostly) work:

1. Swap in the name of one of our SSH keypairs. That's simple and could be documented for Seattle IT.
2. Swap in the ID for an AMI owned by us and also for a security group owned by us. Originally, I did the GIN Project Starter AMI 2019-B.

The Security Group seems easy to document. The main issue is figuring out what we can do to make the AMI requirements as clear and minimal as possible.

I wanted to find a publicly available plain-vanilla Linux AMI that we know this will run on.

I asked Eric B at Stamen what the AMI was and his memory was foggy -- just a basic Amazon Linux AMI. My hunch is that it was this one, which is no longer supported. There are a ton of options that we can peruse in the AWS console GUI, based on this documentation.

Using those docs, I located this Amazon provided Amazon Linux AMI id, ami-03af6a70ccd8cb578. Based on this line in the CloudFormation file, I suspect we should pick Amazon Linux instead of our usual Ubuntu.

Seems like we still might want to modify the Bash script to

1. Check if user exists and if not, create it
2. Make sure that any other dependencies are installed by the script and not assumed to be present

┆Issue is synchronized with this Asana task

[danrademacher]

OK, well, using the Amazon Linux got us a bit further, I can SSH in and the ec2-user exists. But there's no magic website at the IP of the server. https://3.101.109.93/

I think the bash script didn't run, since the first thing is sudo yum update and when I ssh'd in, I saw:

6 package(s) needed for security, out of 16 available
Run "sudo yum update" to apply all updates.

I ran a few commands by hand, but will need to leave this for Tom when he returns to see if we can just run that whole script first by hand and thus understand what we need to do to automate. There were times when I had to agree to downloads, so maybe we just need to override such alerts.

[danrademacher]

Since the Cloudformation takes a while to run each time, we are just going to focus on getting each of these to run: https://github.com/GreenInfo-Network/seattle-building-dashboard/blob/master/pdf-generator/cloudformation.json#L75-L102

Once those are running, we'll rerun the Cloudformation from scratch and make sure it works. At that point, then it would be a matter of highlighting the elements of the file that are account specific

• Keypair Name in the AWS command (not in VC)
• SubnetId and SecurityGroupIds in the CloudFormation JSON file.
• Update SES settings to add any email addresses for users who will be interacting with this https://us-west-1.console.aws.amazon.com/sesv2/home#/account

[tomay]

Here's the general form of the command to connect to the running instance (swap in your .pem and the IP address of the running instance) :

ssh -i PATH_TO_YOUR_PEM_FILE ec2-user@ec2-54-215-253-25.us-west-1.compute.amazonaws.com

[tomay]

After adding a couple of -y to the Bash script, I got "CREATE_COMPLETE"

Note that the AMI that is needed is not found in US-West-2 apparently

[tomay]

Note: Errors in PDF generation will fail silently, and the email will never be sent. Some tips for troubleshooting:

• Stop the server with pm2 stop server.js and start an instance with node server.js. This will then console.log progress and errors
• When everything looks good Ctrl+C to stop the node process, and restart as before with pm2 start server.js

[tomay]

To get the bucket permissions correct, we also had to add a bucket policy to cloudformation.json:

    "BucketPolicy": {
      "Type": "AWS::S3::BucketPolicy",
      "Properties": {
        "PolicyDocument": {
          "Id": "seattle-energy-2021-s3-bucket-policy",
          "Version": "2012-10-17",
          "Statement": [
            {
              "Sid": "PublicReadForGetBucketObjects",
              "Effect": "Allow",
              "Principal": "*",
              "Action": "s3:GetObject",
              "Resource": {
                "Fn::Join": [
                  "",
                  [
                    "arn:aws:s3:::",
                    {
                      "Ref": "S3Bucket"
                    },
                    "/*"
                  ]
                ]
              }
            }
          ]
        },
        "Bucket": {
          "Ref": "S3Bucket"
        }
      }
    }