Introduce New Deployment and Management Tools

[ericwestfall]

Summary

This change introduces new tools and libraries that improve the customer experience for building, testing, deploying, and updating the Terraform Reference Engine in their AWS environment. These changes reduce prerequisite and dependency requirements, enhance resiliency to dependency violation and resource replacement errors, and provide customers with an example Terraform product that can be easily deployed and tested with Terraform Reference Engine.

All changes and features have been tested repeatedly in both fresh AWS environments and environments where Terraform Reference Engine was previously deployed. The automated deployment tooling was also successfully tested with a version of Terraform Reference Engine with the legacy security group and confirmed the security group replacement completed successfully with no user intervention required.

Documentation has been updated in the main project README, and a new tools README has been added.

Tools

A new tools directory has been added which introduces two new command line tools: deploy-terraform-engine.sh and manage-terraform-engine.py.

The deploy-terraform-engine.sh tool enables users to easily deploy the Terraform Reference Engine (TRE) in their AWS environment or safely update a previously deployed TRE environment:

• 🐍 Automatically configures a Python virtual environment, installs and upgrades all required packages, and ensure the virtual environment is activated.
• ☁️ Dynamically determines the AWS account and region in which it is operating.
• 🚧 Checks if the Terraform Reference Engine bootstrap resources have been deployed in the AWS environment and automatically deploys them through a CloudFormation Stack if not.
• 📦 Builds the Terraform Reference Engine packages and deploys artifacts to S3 for staging.
• 🦺 Automatically determines if the Terraform Reference Engine has been previously deployed in the target AWS environment, and if so, gracefully stops the Terraform Reference Engine. [^1]
• 🚧 Deploys the Terraform Reference Engine in the AWS environment.
• 🦺 If updating a previous deployment of the Terraform Reference Engine, the script will automatically re-enable the Terraform Reference Engine and resume request processing. [^1]

The manage-terraform-engine.py tool enables users to safely and easily manage the status of the Terraform Reference Engine deployed in their AWS environment:

• 🔎 Automatically determines the current operating status (e.g., Enabled, Disabled) of the Terraform Reference Engine in an AWS environment and dynamically determines if a user requested action is necessary.
• 🦺 Requires interactive approval from the user before taking any action that will change the operating status of the Terraform Reference Engine, while providing an option to override the approval requirement.
• ⛔️ Exposes a stop command which pauses the processing of new requests, safely terminates all TRE execution instances after allowing in-flight Terraform operations to complete, and prevent launch of new TRE execution instances until re-enabled. [^1]
• 💨 Exposes a start command which re-enables suspended scaling processes, waits for the desired number of running and healthy TRE execution instances, and resumes processing of new requests. [^1]

See the tools documentation for additional detail.

[^1]: These enhancements allows the tooling to automatically handle dependency violation and resource replacement errors (e.g., security group replacement) without depending on manual user intervention or lucky timing.

Containerized Builds

Build operations executed by AWS SAM for the various Terraform Reference Engine functions, including both Python and Go runtimes, have been converted to use build containers, removing the need for users to worry about Python and Go prerequisites in their local environment.

Terraform Runner

The terraform_runner package metadata has been migrated to pyproject.toml and the package is now compliant with PEP 518, PEP 621 and PEP 660. A version 1.0.0 tag has been set in the package and support added for dynamically determining version during build.

Example Terraform Product

A new example directory has been added which includes an example Terraform product and accompanying automation tooling that allows customers to easily deploy and test using the Terraform Reference Engine.

Makefiles

A project level Makefile has been added that:

• 🦺 Automatically validates user environments meet all prerequisite requirements.
• 📦 Builds the various Terraform Reference Engine packages and functions in a consistent and reliable fashion.

A Makefile has been added to the example directory that:

• 🦺 Automatically validates user environments meet all prerequisite requirements to build and deploy the Terraform Example product.
• 📦 Builds the Terraform Example provisioning artifact and deploys it to the staging S3 bucket.
• 🚧 Automates the creation of the Service Catalog product, portfolio, association, and launch constraint.
• 🚧 Automates the creation of IAM roles required for the launch role.
• ✈️ Provides convenience targets that allow users to easily share the Service Catalog portfolio with an AWS Organizations OU and grant access to IAM security principals.

[smaly-amazon]

``The overall intent of the PR is great. having better deploy and management tools will be a nice improvement.

Here are a few preliminary comments after I pulled the PR locally and tried deploying it.

---

Please remove the sample-provisioning-artifacts directory. You've replaced this with your new example.

---

When I run make check-prerequisites:

Makefile:98: *** [ERROR] Unable to determine the correct AWS region, please ensure your desired region is properly defined in your environment. Stop.

I have my region set in ~/.aws/config, which the makefile doesn't recognize.

Later in the instructions for ./tools/deploy-terraform-engine.sh there is an explanation of setting env vars.

All was fine with make after I set AWS_REGION env var. Can we move the details about env vars up to the instructions for running make check-prerequisites?

---

While running ./tools/deploy-terraform-engine.sh --region:

Error: Docker is unreachable. Docker needs to be running to build inside a container. make: *** [build-lambda-functions] Error 1

But docker seems to be running, and when I run again make check-prerequisites:

All prerequisites are installed and configured properly.

I've never built sam using containers, so maybe there's still some prereq missing here for people installing all the tools for the first time?

[userhas404d]

Thanks for this! If you're looking for more data points I was able to run deploy-terraform-engine.sh successfully (macos) but did have to run it twice after hitting the following error:

Error: Failed to package template: /<some path>/service-catalog-engine-for-terraform-os/tools/.aws-sam/build/template.yaml. 
 [Errno 2] No such file or directory: '/<some path>/service-catalog-engine-for-terraform-os/.aws-sam/packaged.yaml'
make: *** [build-lambda-functions] Error 1
2023-09-15T19:50:06Z [ERROR] - An error occurred while attempting to execute the build-lambda-functions target in makefile at path /<some path>/service-catalog-engine-for-terraform-os/Makefile

Not sure if this is important but on the first pass (the one that raised the error above) I invoked deploy-terraform-engine.sh from within the tools dir, and on the second (successful) invoke, from the project root.

Also not super relevant to any of this but I think its kinda funny that you're orchestrating the deployment of a terraform oriented tool independent of terraform itself. Not meant to be a dig, just ironic. In any case I appreciate how straightforward of a process this deploy was with these additions.