Add support for Ec2 instance deployment

pditommaso commented 1 year ago

One requirement for the automated deployment is the ability to deploy Seqera Platform is a single VM environment. In this scenario the Terraform module should be able to:

Deploy the RDS database
Deploy the Redis Elasticsearch instance
Create an IAM role with this policy named seqera-forge-role
Spin an Ec2 instance that will run a single node K3s cluster using seqera-forge-role as instance tole
Configure the security group allowing inbound traffic on port 25 22 (ssh), 80, 443

enekui commented 1 year ago

I am almost done with all the part at logic level that was blocking me. We will need to have a call to clarify a lot details regarding the EC2 instance and many others regarding the AWS Loadbalnacer. There is a lot pending to be clarified.

pditommaso commented 1 year ago

Let's focus on. the Ec2 instance, instance role and security group. It would be a great start

enekui commented 1 year ago

Let's focus on. the Ec2 instance, instance role and security group. It would be a great start

Which AMI to use for the EC2 instance?
How the instance will be accessed, SSH, SSMSessionManager?
Default instance type
Associate Public IP?
EBS volume type
StartUp Script?

pditommaso commented 1 year ago

Which AMI to use for the EC2 instance?

a parameter defaulting defaulting to Amazon linux AMI

How the instance will be accessed, SSH, SSMSessionManager?

SSH

Default instance type

The same as for EKS node instance type

Associate Public IP?

Optional, default YES

EBS volume type

gp3, size defined via a variable default to 100GB

StartUp Script?

No (at least for now)

enekui commented 1 year ago

Updates

Refactored VPC networking stack. Now the customer don't need to interact with the networking design, it's all computed in runtime by Terraform using the cidrsubnet fuction in combination with other functions. This improvement also allows that availability zones to be computed by Terraform.
Added the capability to deploy the EC2 instance in a public network and get public IP, this had a new chanllenge since the RDS and Redis are always deployed to private networks, to resolve this I added support for VPC endpoints that will allow this communication flow from the EC2 public network using VPC endpoints.
Added support for SSMSessionManager as a connectivity solution to the EC2 instance, since when the instance is deployed to private networks SSH won't work.
Added support to auto-generate ssh key in the cloud from local SSH key material.
Added variable validation for the VPC cidr, and num of availablity zones to keep this under control.
Updated README.md
Added other small improvements to code.

Example of public EC2 instance `main.tf`

module "terraform-seqera-module" {
  source  = "github.com/seqeralabs/terraform-seqera-aws"
  aws_profile = "development"
  region  = "eu-west-2"

  ## VPC
  vpc_name = "seqera-vpc"

  ## EC2 Instance
  create_ec2_instance = true
  create_ec2_instance_local_key_pair = true
  create_public_ec2_instance = true

  default_tags = {
    Environment = "development"
    ManagedBy   = "Terraform"
    Product     = "Seqera"
    CreatedBy   = "DevOps"
  }
}

output "database_url" {
  value = module.terraform-seqera-module.database_url
}

output "redis_url" {
  value = module.terraform-seqera-module.redis_url
}

Example of private EC2 instance

module "terraform-seqera-module" {
  source  = "github.com/seqeralabs/terraform-seqera-aws"
  aws_profile = "development"
  region  = "eu-west-2"

  ## VPC
  vpc_name = "seqera-vpc"

  ## EC2 Instance
  create_ec2_instance = true
  enable_ec2_instance_session_manager_access = true

  default_tags = {
    Environment = "development"
    ManagedBy   = "Terraform"
    Product     = "Seqera"
    CreatedBy   = "DevOps"
  }
}

output "database_url" {
  value = module.terraform-seqera-module.database_url
}

output "redis_url" {
  value = module.terraform-seqera-module.redis_url
}

Example of public EC2 instance with secure SSM Session Manager access `main.tf`

module "terraform-seqera-module" {
  source  = "github.com/seqeralabs/terraform-seqera-aws"
  aws_profile = "development"
  region  = "eu-west-2"

  ## VPC
  vpc_name = "seqera-vpc"

  ## EC2 Instance
  create_ec2_instance = true
  enable_ec2_instance_session_manager_access = true
  create_public_ec2_instance = true
  ec2_instance_security_group_ingress_rules_names = ["http-80-tcp", "https-443-tcp"]

  default_tags = {
    Environment = "development"
    ManagedBy   = "Terraform"
    Product     = "Seqera"
    CreatedBy   = "DevOps"
  }
}

output "database_url" {
  value = module.terraform-seqera-module.database_url
}

output "redis_url" {
  value = module.terraform-seqera-module.redis_url
}

Commit

f7e9df189943aaab2fcbc61bba91724146304428

pditommaso commented 1 year ago

Nice, please make a PR for this

enekui commented 1 year ago

Nice, please make a PR for this

Sure! But I need to continue doing some more testing before that. I am testing all the different possibilities, and scenarios. Will improve code comments, will update README.md, etc.

pditommaso commented 1 year ago

A PR can be in draft mode, no worries about not being complete

seqeralabs / terraform-seqera-aws