PowerDataHub / terraform-aws-airflow

Terraform module to deploy an Apache Airflow cluster on AWS, backed by RDS PostgreSQL for metadata, S3 for logs and SQS as message broker with CeleryExecutor
Apache License 2.0
84 stars 40 forks source link

Specify vpc_id failed when creating RDS instance #18

Closed AndyHuang1991 closed 5 years ago

AndyHuang1991 commented 5 years ago

Module Version: 0.7.2 (after initial investigate, this issue might also happened in latest version 0.8.10)

Issue: when I providing vpc_id variables (my existing vpc). it will failed when creating RDS instance.

Reason: this module will create rds in default vpc, and with providing VPC id, it will generate and attached providing VPC's secruity group to default VPC which cause failure.

The vpc_security_group_ids in bellowed code (in main.tf)

resource "aws_db_instance" "airflow_database" {
  identifier = "${module.airflow_labels.id}-db"
  allocated_storage = "${var.db_allocated_storage}"
  engine = "postgres"
  engine_version = "11.1"
  instance_class = "${var.db_instance_type}"
  name = "${var.db_dbname}"
  username = "${var.db_username}"
  password = "${var.db_password}"
  storage_type = "gp2"
  backup_retention_period = 14
  multi_az = false
  publicly_accessible = false
  apply_immediately = true
  skip_final_snapshot = true
  vpc_security_group_ids = ["${module.sg_database.this_security_group_id}"]
  port = "5432"
}

Possible solution: create subnet group with providing vpc and add to aws_db_instance's db_sunet_group argument

edbizarro commented 5 years ago

hi @AndyHuang1991

I'll look into it!

Thanks

asafcombo commented 5 years ago

Issue still persists in version 0.9.1

error:

Error: Error creating DB Instance: InvalidParameterCombination: The DB instance and EC2 security group are in different VPCs. The DB instance is in vpc-XXYY and the EC2 security group is in vpc-ZZWW
        status code: 400, request id: 1420c07e-4a42-41e0-9b88-1a70a9c03b9f

  on .terraform/modules/airflow/PowerDataHub-terraform-aws-airflow-a679957/main.tf line 411, in resource "aws_db_instance" "airflow_database":
 411: resource "aws_db_instance" "airflow_database" {