Deploy with VPC endpoint

Summary

OpenSearch VPC endpoint cannot be created manually before SIEM platform deployment.

Details

While deploying the stack (through Terraform) with a provided VpcEndpointId (in order to configure the SIEM platform as private), we fall in a deadlock situation.

Here is the Terraform code:

resource "aws_cloudformation_stack" "siem" {
  name = "${var.project_id}-siem-${var.aws_region}"
  parameters = {
    AllowedSourceIpAddresses = "10.0.0.0/8 172.16.0.0/12 192.168.0.0/16"
    DeploymentTarget = "opensearch_managed_cluster"
    DomainOrCollectionName = "${var.project_id}-siem-${var.aws_region}"
    SnsEmail = var.sns_email
    ReservedConcurrency = 50
    VpcEndpointId = aws_opensearch_vpc_endpoint.siem_vpc_endpoint.id
    CreateS3VpcEndpoint = true
    CreateSqsVpcEndpoint = true
    CreateSsmVpcEndpoint = true
    CreateStsVpcEndpoint = true
  }
  template_url = "https://aes-siem-${var.aws_region}.s3.amazonaws.com/siem-on-amazon-opensearch-service.template"
  capabilities = [ "CAPABILITY_NAMED_IAM" ]
  timeouts {
    create = "60m"
  }
}

data "aws_subnets" "private" {
  filter {
    name   = "vpc-id"
    values = [local.shared_vpc_id]
  }

  filter {
    name   = "tag:Type"
    values = ["private"] # insert values here
  }
}

resource "aws_security_group" "siem" {
  name        = "${var.project_id}-siem"
  description = "Opensearch SIEM SG"
  vpc_id      = local.shared_vpc_id

  ingress {
    from_port = 443
    to_port   = 443
    protocol  = "tcp"
    prefix_list_ids = [local.admins_prefix_list_id]
  }

  egress {
    from_port = 443
    to_port   = 443
    protocol  = "tcp"
    prefix_list_ids = [local.admins_prefix_list_id]
  }
}

resource "aws_opensearch_vpc_endpoint" "siem_vpc_endpoint" {
  domain_arn = "arn:aws:es:${var.aws_region}:${var.aws_account_id}:domain/${var.project_id}-siem-${var.aws_region}"
  vpc_options {
    security_group_ids = [aws_security_group.siem.id]
    subnet_ids         = data.aws_subnets.private.ids
  }
}

We are facing this error: Error: creating OpenSearch VPC Endpoint: ValidationException: Either the domain doesn't exist, it doesn't support creation of VPC endpoints

Cause of error

This is because it's not possible to create the OpenSearch VPC endpoint before creating the OpenSearch platform itself.

Version

v2.10.2a

Possible solutions

Solution 1

Integrate the VPC endpoint creation directly in the CloudFormation template.

Solution 2

In the CloudFormation template, add a parameter like PubliclyAccessible (true/false) to determine the type of access, so that we can attach a VPC endpoint a posteriori.

In your CloudFormation template, this part:

  hasVpce: !Not
    - !Equals
      - !Ref 'VpcEndpointId'
      - ''
  IsInVpc: !Or
    - !Equals
      - false
      - true
    - !Condition 'hasVpce'

would be replaced by something like:

  IsInVpc: !Equals [!Ref PubliclyAccessible, false]

aws-samples / siem-on-amazon-opensearch-service