TLS-enable service.civicpdx.org

[MikeTheCanuck]

We have issued a TLS certificate from ACM (Amazon Certificate Manager).

We have a stretch goal to TLS-enable service.civicpdx.org so that all endpoints would be https'd, e.g. https://service.civicpdx.org/disaster-resilience/api/Address/ rather than http://service.civicpdx.org/disaster-resilience/api/Address/

[MikeTheCanuck]

I believe this is the primary reference for making this work: https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-create-https-ssl-load-balancer.html

Since we're load-balancing multiple containers for each route at service.civicpdx.org, there's literally no way to terminate the SSL connections inside the containers. Instead they'll have to terminate at the load balancer. For our purposes, there's no requirement to encrypt traffic from the load balancer to the containers, so we'll just leave that path running over http/80.

There's some clues here as well, about configuring Policies that set TLS properties for the TLS listener - but since this is the v1 ELB configuration it's not clear at what level of the hierarchy the Policies object will go in a v2 configuration (since v1 and v2 have the Listeners at a different level of the hierarchy as well): https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-elb.html

[MikeTheCanuck]

Found this example: https://github.com/aws-samples/aws-refarch-drupal/blob/master/templates/aws-refarch-drupal-03-publicalb.yaml

With this little tidbit:

Use the ACM certificate Amazon Resource Name (ARN) as the optional Cloudfront and/or Public ALB ACM certificate input parameters of the master template.

Doesn't say where to drop that ARN in or anything, but hey bro, thanks for at least half a clue.

[MikeTheCanuck]

With commit 9311abaa304beaed9f92b019061810abf94a3645, we again have templates that deploy successfully (change set passes linting and deployment).

However, there are at least two issues remaining:

1. The listener for HTTPS/443 was not deployed
2. I have no earthly idea where to stuff the ARN for the certificate - which may be one reason why the listener didn't deploy.

Odd that something wouldn't deploy as specified, but allow the changeset to execute to completion.

[MikeTheCanuck]

Even after deploying updated CloudFormation templates that include an explicit reference to the certificate's ARN, the 443 listener was not created, so I remain stumped.

I'm digging further into other example ALB templates, AWS docs on SSL load balancers and ELB configuration. God help me if I have to read through all the CloudFormation documentation on how to create Templates and properly implement Parameters - I can definitely say that's far more deep knowledge than I ever intended to gain for this bespoke technology.

[MikeTheCanuck]

My mind starts asking hard questions when I really start digging into the weeds of CF templates like these two that I'm attempting to manipulate:

1. Is it possible for parameters in these templates to derive their specific value from outside the templates - e.g. from some runtime pass-through from AWS (e.g. from some bootstrapped values in ECS, or something that was generated as a result of CloudFormation creating and maintaining this stack)? I'm thinking of things like !Ref AWS::StackName that I haven't tracked down yet.
2. In which template and location should I populate the value for PublicAlbAcmCertificate? Or is that also something that CloudFormation or ECS will pull in by matching the intended FQDN being routed, with a matching cert in ACM?

[MikeTheCanuck]

I'm roaming around the AWS CF documentation and noticed "SslPolicy" on this page: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticloadbalancingv2-listener.html

It's marked as "Required: No", not even "Conditional" so it's possible that CF just uses some intrinsic defaults if there's nothing explicitly declared. Certainly all the templates I'm reviewing don't mention it.

[MikeTheCanuck]

OK, so progress made so far:

• with help from @DingoEatingFuzz, now have successfully created the 443/HTTPS Listener in EC2 via Cloudformation Templates

Next up is to bind the endpoint-service container to the 443/HTTPS Listener so that it ALB will forward both http://service.civicpdx.org:80/ and https://service.civicpdx.org:443/ traffic to the container listening on 8000/HTTP.

I'm working this current problem in PR 52, and as of this commit, CloudFormation is giving me an error "Parameters: [ListenerTls] must have values".

Here's what I've reasoned (properly or not) so far:

• I've added the Output ListenerTls to load-balancers.yaml, sending the object LoadBalancerListenerTls
• In services/endpoint-service/service.yaml I've declared a Parameter ListenerTls, and I'm consuming it in the new ListenerRule I've added here

What I've tried so far:

• changed the Priority value for these two ListenerRules to be the highest-ranked, per Ian's comment in this doc that "The civic-lab service must be the highest priority number of all services (last rule to be evaluated before the catch-all default rule) since it's Path is ( / )". NOTE: now that I've replaced civic-lab service with endpoint-service to respond at "/", there's every reason to believe that I've done the right thing here. This change (from low Priority values to very high values) eliminated extra errors during execution of the changeset that reported "The following resource(s) failed to update: [EndPoints, 2018TS, HousingService, 2018LE]" and now just reports "The following resource(s) failed to update: [EndPoints]"
• changed the name of ListenerRule to ListenerRule2 - same error

I have a few remaining-but-bad theories on what's causing the error:

1. The ListenerArn is receiving a reference to the ListenerTls object, and instead we somehow need to pass it a reference to the Arn property of ListenerTls - but this seems unnecessary as the original ListenerRule makes the same references, seemingly without a problem
2. The Output of ListenerTls in the load-balancers.yaml is generating an object that can't be consumed by service.yaml as the String type that is currently declared - but this seems unlikely, as the original Listener object Output appears just as complex an object, and it's being consumed as a String type just fine.
3. Might need to create a new TargetGroup that's configured for Port: 443 and Protocol: HTTPS - I keep going back and forth thinking the ListenerRules should be fine forwarding their traffic to the same TargetGroup, but then I realize that the application is listening on 8000, and then I get confused and can't figure what role the TargetGroup plays here.

[MikeTheCanuck]

PR 52 has nailed the key problem: getting a container to respond to requests on 80/http and 443/https.

Now that this has been accomplished for endpoint-service listening on "/", we can replicate this pattern for the rest of the 2018 containers that are responding to the API routes.

[MikeTheCanuck]

2018 APIs: https://github.com/hackoregon/hackoregon-aws-infrastructure/pull/53 Frontend container services: https://github.com/hackoregon/hackoregon-aws-infrastructure/pull/54 2017 APIs: https://github.com/hackoregon/hackoregon-aws-infrastructure/pull/57

All done - everything can answer to either http:// or https:// (though the return data needs some attention to make it less hard-coded to http:// URLs). That's for another issue.