To launch pipelines with Seqera Platform Enterprise, you first need to deploy a Platform instance. A successful deployment requires you to make decisions about networking, security posture, container orchestration, permissions, etc. These decisions can be overwhelming to a client administrator who just wants to get their users launching Nextflow pipelines from the platform.
The Seqera CX team has developed a field tool to help simplify these initial efforts in two ways:
Reduce the infrastructure and configuration burden, so that a first-time deployment of Seqera Platform Enterprise requires less than 1 hour of work and only a few minutes for subsequent redeployments.
Provide an Infrastructure-As-Code (IaC) solution so activities are repeatable and artifacts can be checked into source-control.
This is an unofficial field tool.
The solution is delivered on a best-effort basis, but provides no guarantees of appropriateness for your specific scenario. Please conduct your due diligence prior to execution within your environment.
The out-of-the-box solution depends on you meeting the following criteria:
~/.ssh/config
file. Install terraform v1.3.7
or later:
Install and configure the latest aws cli
(version 2):
Install a modern version of openssh
:
Install git v2.25.1
or later.
Install GNU Make 4.2.1
or later.
Install python 3.8.10
or later.
Please see Permissions for a full list of necessary AWS IAM permissions required to run the tool.
Please see Security for details on how we scan for / mitigate tool security vulnerabilities.
The tool relies on four core configuration files, all stored within the templates
folder. These files must be prepared prior to project execution:
terraform.tfvars
Used to set flags and values that control how your infrastructure and the Seqera Platform Enterprise instance is built and configured.
This file must be configured and stored in the project folder root.
ssm_sensitive_values_tower.json
Used to define secrets like credentials and passwords which the Seqera Platform needs to fulfill its duties.
This file exists to make SSM data population more convenient and should not be stored in source-control.
ssm_sensitive_values_seqerakit.json
(Optional)
Used to define secrets like tokens and credentials which the Seqerakit tool needs to fulfill its duties.
This file exists to make SSM data population more convenient and should not be stored in source-control.
ssm_sensitive_values_groundswell.json
(Optional)
Used to define credentials for the Groundswell optimization service.
This file exists to make SSM data population more convenient and should not be stored in source-control.
Download a copy of the repository to your local workstation:
git clone <TODO_path_to_offical_repo> && cd <TODO_name_of_directory>
Select a name for your Seqera Platform application (default: tower-dev
).
This is a namespace isolator which prevents concurrent instances (e.g., dev
and prod
) from accidentally sharing configurations.
Modify the ssm_sensitive_values_tower.json
in the templates
folder:
Perform a find-and-replace on tower-dev
if you select a different application name.
Replace every instance of CHANGE_ME
in the file with the values supplied to you during your Seqera onboarding.
Replace every instance of CHANGE_ME_SEE_SEQERA_DOCS
with your own values. Configuration value guidance can be found at Seqera Platform Enterprise configuration.
Review each instance of CHANGE_ME_IF_NECESSARY
and replace as needed.
WARNING:
ssm_key
entries apart from the application name. Path changes will prevent the application from booting.Modify the ssm_sensitive_values_seqerakit.json
in the templates
folder:
Execute a find-and-replace on tower-dev
if you select a different application name.
Replace every instance of CHANGE_ME
in the file with the values appropriate for your organization.
Review each instance of CHANGE_ME_IF_NECESSARY
and replace as needed.
An AWS credential set is a mandatory minimum. The terraform.tfvars
file has flags that can enable/disable expectations regarding the AWS IAM role, Github credential, and Docker credential.
WARNING:
ssm_key
entries apart from the application name. Path changes will prevent the application from booting.Modify the ssm_sensitive_values_groundswell.json
in the templates
folder:
Execute a find-and-replace on tower-dev
if you select a different application name.
Replace default values if necessary.
Decide on the AWS Parameter Store prefixes where you will store the payloads of your now-populated ssm_*
files.
Default values in terraform.tfvars
are:
/seqera/sensitive_values/tower_dev/tower
/seqera/sensitive_values/tower_dev/seqerakit
/seqera/sensitive_values/tower_dev/groundswell
Create the entries and paste the appropriate JSON payload for each. Ensure that:
SecureString
.text
.Remember your application name and SSM prefixes, as these need to be supplied during the configuration of the terrform.tfvars
file.
terraform.tfvars
fileNOTE: Fulsome field-level documentation exists inside the terraform.tfvars
file. Consult there for details beyond the high-level instructions here.
Create a working copy of the terraform.tfvars
file in the project root.
# In project root
cp templates/TEMPLATE_terraform.tfvars terraform.tfvars
Update the app_name
, secrets_bootstrap_tower
, secrets_bootstrap_seqerakit
, and secrets_bootstrap_groundswell
with the values you selected during the secrets preparation phase.
Replace every instance of REPLACE_ME
with values appropriate for your organization.
Set flag_*
variables to true
/ false
to match your desired implementation architecture.
Review each instance of REPLACE_ME_IF_NECESSARY
and modify as required (these decisions will be driven by the flag activations).
Review and modify placehoder values as necessary. Do not modify commented fields with DO_NOT_UNCOMMENT_ME
values. These are in the file solely to provide a visual reminder that these keys are being set behind the scene via SSM secrets values.
Modify the JSON block in the documentation
folder:
Perform a find-and-replace on APP_NAME_REPLACE_ME
(suggested default is tower-dev
).
Replace every AWS_REGION_REPLACE_ME
with your desired AWS region.
Replace every AWS_ACCOUNT_REPLACE_ME
instance with the id of the AWS Account which the Terraform installer will interact with.
Create an AWS IAM Role / User and attach these permissions.
Configure this identity into your AWS CLI and ensure it is active when executing the installer (specified in var.aws_profile
).
During the installation process, a SSH config file is created and used to connect to the EC2 instance hosting the Seqera Platform instance. To use the SSH config file successfully, you need a minor modification to your openssh
configuration:
# Add the following entry AT THE TOP of your ~/.ssh/config file
Include /ABSOLUTE_PATH_TO_INSTALLER_PROJECT_ROOT/ssh_config
# NOTE: IF you intend to run multiple instances on the same host (e.g. dev and staging), add an Include to each project folder.
Include /ABSOLUTE_PATH_TO_INSTALLER_PROJECT_ROOT_DEV/ssh_config
Include /ABSOLUTE_PATH_TO_INSTALLER_PROJECT_ROOT_STAGING/ssh_config
The project ships with a .githooks
folder, which contains a Python script that wil scan your terraform.tfvars
file for configuration mismatches which cause your deployment to fail.
To automatically invoke the script prior to a commit to your git repository, execute the following in the root of your project: git config core.hooksPath .githooks
.
By default, the installer writes the Terraform state to a local folder (<PROJECT_ROOT>/DONTDELETE
). This is convenient for initial testing but likely not a good long-term solution for most clients.
You can change the state management strategy at the top of the 000-main.tf
file. We have provided a commented-out example that demonstrates how to write state to an S3 bucket.
Note: Bulletproof state management can become complex and is beyond the scope of this initiative. Consult Terraform's official docs on Remote State and modify your project as necessary to meet your organization's needs.
Within your terminal, navigate to the project root and initialize the project:
terraform init
Create and review an execution plan:
# Recommended approach. Execute the Seqera-supplied Python script to check your `terraform.tfvars` file for known configuration conflicts prior to terraform binary invocation.
make plan
# Alternative approach to execute plan without Python script verification execution.
terraform plan
Execute the actions reviewed in the Terraform plan:
# Recommended approach. Execute the Seqera-supplied Python script to check your `terraform.tfvars` file for known configuration conflicts prior to terraform binary invocation.
make apply
# Alternative approach to execute plan without Python script verification execution.
# Note: You can append `--auto-append` to the end of the command to avoid the need to type 'yes' to approve the deployment.
terraform apply
terraform destroy
The tool offers the ability to configure the specific docker logging driver used to capture Tower container logs. By default, the local file
logging driver is configured.
If a database (regardless if container or RDS) was created as part of the deployment, teardown will destroy it and all data within.
Prior to deletion, consider backing up your database if the data may be needed in future.
Terraform is not aware of actions executed within the Seqera Platform (i.e., invocation of Tower Forge to create compute environments by users / automation tools like Seqerakit). Executing terraform destroy
without first conducting a purge of objects within Seqera Platform will result in orphaned assets in your AWS Account.
Given client environment variability, Seqera offers no official guidance re: how best to run multiple concurrent implementations (e.g. entirely separate repositories, different branches in a monorepo, Terraform workspaces, git submodules, etc). Each site must decide what is best for them and implement accordingly.
With that said, for design purposes, this tool assumes that multiple project instances will live in the same filesystem (each within its own exclusive namespace). Each project's ssh_config
file uses an alias matching the unique app_name
from your tfvars
file, making it possible to add multiple non-conflicting Include
statements in your ~/.ssh/config
.
Name | Version |
---|---|
aws | 5.12.0 |
random | 3.6.0 |
null | 3.2.2 |
tls | 4.0.5 |
template | 2.2.0 |
Name | Description |
---|---|
aws_account_id | AWS Account ID |
aws_caller_arn | Assumed role used to deploy the project |
aws_caller_user | User used to deploy the project |
ec2_ssh_key | SSH key attached to the EC2 instance |
tower_server_url | URL to check your Tower intance |
route53_record_status | Identifies if a Route53 record was created or not |
aws_ec2_private_ip | Private IP of the EC2 Instance |
aws_ec2_public_ip | Public IP of the EC2 Instance (if applicable) |
tower_api_endpoint | Platform API endpoint |
seqera_configuration | Path to the SeqeraKit setup file |
redis_endpoint | URL for your Redis instance |