search

antonbabenko / modules.tf-lambda

Infrastructure as code generator - from visual diagrams created with Cloudcraft.co to Terraform
https://www.cloudcraft.co/
MIT License
351 stars 56 forks source link

Terragrunt Aplly-ALL doesnt work multiple errors #28

Open Genaker opened 4 years ago

Genaker commented 4 years ago 
[terragrunt] [/home/genaker/Downloads/Magento-Terraform-Infrastructure(3)/magento-terraform-infrastructure/us-west-1/mysql-master-rds] 2020/03/24 22:18:39 Running command: terraform apply -input=false -input=false -auto-approve
module.db_instance.data.aws_iam_policy_document.enhanced_monitoring: Refreshing state...

Error: first character of "name_prefix" must be a letter

  on modules/db_option_group/main.tf line 1, in resource "aws_db_option_group" "this":
   1: resource "aws_db_option_group" "this" {

Error: only alphanumeric characters and hyphens allowed in "name_prefix"

  on modules/db_option_group/main.tf line 1, in resource "aws_db_option_group" "this":
   1: resource "aws_db_option_group" "this" {

Error: only lowercase alphanumeric characters and hyphens allowed in parameter group "name_prefix"

  on modules/db_parameter_group/main.tf line 33, in resource "aws_db_parameter_group" "this":
  33: resource "aws_db_parameter_group" "this" {

Error: first character of parameter group "name_prefix" must be a letter

  on modules/db_parameter_group/main.tf line 33, in resource "aws_db_parameter_group" "this":
  33: resource "aws_db_parameter_group" "this" {

aws_lb.this[0]: Creating...
[terragrunt] [/home/genaker/Downloads/Magento-Terraform-Infrastructure(3)/magento-terraform-infrastructure/us-west-1/mysql-master-rds] 2020/03/24 22:18:43 Module /home/genaker/Downloads/Magento-Terraform-Infrastructure(3)/magento-terraform-infrastructure/us-west-1/mysql-master-rds has finished with an error: Hit multiple errors:
exit status 1

Error: Error creating application Load Balancer: ValidationError: At least two subnets in two different Availability Zones must be specified
    status code: 400, request id: bee0f9bb-e951-41f9-9ea0-969803b89e32

  on main.tf line 1, in resource "aws_lb" "this":
   1: resource "aws_lb" "this" {
Genaker commented 4 years ago 

aws_lb.this[0]: Creating...
2020/03/24 22:41:10 [DEBUG] aws_lb.this[0]: applying the planned Create change
2020-03-24T22:41:10.923-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: 2020/03/24 22:41:10 [DEBUG] setting computed for "subnets" from ComputedKeys
2020-03-24T22:41:10.923-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: 2020/03/24 22:41:10 [DEBUG] setting computed for "security_groups" from ComputedKeys
2020-03-24T22:41:10.923-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: 2020/03/24 22:41:10 [DEBUG] setting computed for "subnet_mapping" from ComputedKeys
2020-03-24T22:41:10.923-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: 2020/03/24 22:41:10 [DEBUG] ALB create configuration: {
2020-03-24T22:41:10.923-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4:   IpAddressType: "ipv4",
2020-03-24T22:41:10.923-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4:   Name: "mutual-foal",
2020-03-24T22:41:10.923-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4:   Tags: [{
2020-03-24T22:41:10.923-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4:       Key: "Name",
2020-03-24T22:41:10.923-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4:       Value: "mutual-foal"
2020-03-24T22:41:10.923-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4:     }],
2020-03-24T22:41:10.923-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4:   Type: "application"
2020-03-24T22:41:10.923-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: }
2020-03-24T22:41:10.924-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: 2020/03/24 22:41:10 [DEBUG] [aws-sdk-go] DEBUG: Request elasticloadbalancing/CreateLoadBalancer Details:
2020-03-24T22:41:10.924-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: ---[ REQUEST POST-SIGN ]-----------------------------
2020-03-24T22:41:10.924-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: POST / HTTP/1.1
2020-03-24T22:41:10.924-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: Host: elasticloadbalancing.us-west-1.amazonaws.com
2020-03-24T22:41:10.924-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: User-Agent: aws-sdk-go/1.29.24 (go1.13.7; linux; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.12.24 (+https://www.terraform.io)
2020-03-24T22:41:10.924-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: Content-Length: 152
2020-03-24T22:41:10.924-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: Authorization: AWS4-HMAC-SHA256 Credential=AKIAUZMFRD25Q7PATRG4/20200325/us-west-1/elasticloadbalancing/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date, Signature=db70cb9eb32e73b9650d8dc7a7eb6c0d070dda8b467be9d1091ecc48f2e35901
2020-03-24T22:41:10.924-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: Content-Type: application/x-www-form-urlencoded; charset=utf-8
2020-03-24T22:41:10.924-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: X-Amz-Date: 20200325T054110Z
2020-03-24T22:41:10.924-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: Accept-Encoding: gzip
2020-03-24T22:41:10.924-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: 
2020-03-24T22:41:10.924-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: Action=CreateLoadBalancer&IpAddressType=ipv4&Name=mutual-foal&Tags.member.1.Key=Name&Tags.member.1.Value=mutual-foal&Type=application&Version=2015-12-01
2020-03-24T22:41:10.924-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: -----------------------------------------------------
2020-03-24T22:41:10.950-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: 2020/03/24 22:41:10 [DEBUG] [aws-sdk-go] DEBUG: Response elasticloadbalancing/CreateLoadBalancer Details:
2020-03-24T22:41:10.950-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: ---[ RESPONSE ]--------------------------------------
2020-03-24T22:41:10.950-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: HTTP/1.1 400 Bad Request
2020-03-24T22:41:10.950-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: Connection: close
2020-03-24T22:41:10.950-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: Content-Length: 337
2020-03-24T22:41:10.950-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: Content-Type: text/xml
2020-03-24T22:41:10.950-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: Date: Wed, 25 Mar 2020 05:41:10 GMT
2020-03-24T22:41:10.950-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: X-Amzn-Requestid: 97a7dad9-a48e-4131-8b90-8f82981c0c32
2020-03-24T22:41:10.950-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: 
2020-03-24T22:41:10.950-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: 
2020-03-24T22:41:10.950-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: -----------------------------------------------------
2020-03-24T22:41:10.950-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: 2020/03/24 22:41:10 [DEBUG] [aws-sdk-go] <ErrorResponse xmlns="http://elasticloadbalancing.amazonaws.com/doc/2015-12-01/">
2020-03-24T22:41:10.950-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4:   <Error>
2020-03-24T22:41:10.950-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4:     <Type>Sender</Type>
2020-03-24T22:41:10.950-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4:     <Code>ValidationError</Code>
2020-03-24T22:41:10.950-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4:     <Message>At least two subnets in two different Availability Zones must be specified</Message>
2020-03-24T22:41:10.950-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4:   </Error>
2020-03-24T22:41:10.950-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4:   <RequestId>97a7dad9-a48e-4131-8b90-8f82981c0c32</RequestId>
2020-03-24T22:41:10.950-0700 [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: </ErrorResponse>
``
Genaker commented 4 years ago

The first error is the named validation problem.

Second Elb Module Terraform problem

antonbabenko commented 4 years ago

First assumption is that you have named your DB instance starting with the letter. Text labels are passed into module as argument (name_prefix) without validation (there is WIP issue already - #18).

ALB is failing because you have to specify "At least two subnets in two different Availability Zones must be specified". If you have edited this manually, make sure you set it correctly. Normally VPC resources like subnets are created across all available AZs.

If problem persists, please share a link to your blueprint by email.

Genaker commented 4 years ago

Yes, but we can't manage Az in the cloud craft. It is something that scripts (module.tf) do during the import. Name prefix is 

  # The name of the RDS instance, if omitted, Terraform will assign a random, unique identifier
  # type: string
  identifier = "MySQL-Master-RDS"

name_prefix is not part of terragrunt configuration

Genaker commented 4 years ago

Hi! I still struggling with this error.

Error: first character of "name_prefix" must be a letter
  on modules/db_option_group/main.tf line 1, in resource "aws_db_option_group" "this":
   1: resource "aws_db_option_group" "this" {

Error: only alphanumeric characters and hyphens allowed in "name_prefix"
  on modules/db_option_group/main.tf line 1, in resource "aws_db_option_group" "this":
   1: resource "aws_db_option_group" "this" {

Error: only lowercase alphanumeric characters and hyphens allowed in parameter group "name_prefix"

  on modules/db_parameter_group/main.tf line 33, in resource "aws_db_parameter_group" "this":
  33: resource "aws_db_parameter_group" "this" {

Error: first character of parameter group "name_prefix" must be a letter

  on modules/db_parameter_group/main.tf line 33, in resource "aws_db_parameter_group" "this":
  33: resource "aws_db_parameter_group" "this" {

And the weirdest one is ALB doesn't work 

Error: Error creating application Load Balancer: ValidationError: At least two subnets in two different Availability Zones must be specified
    status code: 400, request id: 3cbf0f7c-24bc-4461-b9db-2a2a91886a49

  on main.tf line 1, in resource "aws_lb" "this":
   1: resource "aws_lb" "this" {

I'm trying to fix parameter name by fix name

antonbabenko commented 4 years ago

RDS identifier is using wrong characters inside but since that value is used to create some other resources you need to update it to satisfy these error messages - "only lowercase alphanumeric characters and hyphens", "first character ... must be a letter".

ALB error is probably related to a missing subnets argument which should look like this:

inputs {
  # ... omitted
  subnets = dependency.vpc.outputs.public_subnets
}

If not, please show inputs in alb directory and I will try to help.

Genaker commented 4 years ago

Yes, there are no subnets = dependency.vpc.outputs.public_subnets


###########################################################
# View all available inputs for this module:
# https://registry.terraform.io/modules/terraform-aws-modules/alb/aws/5.1.0?tab=inputs
###########################################################
inputs = {
  # The resource name and Name tag of the load balancer.
  # type: string
  name = "novel-husky"

}

###########################################################

View all available inputs for this module:

https://registry.terraform.io/modules/terraform-aws-modules/alb/aws/5.1.0?tab=inputs

########################################################### inputs = {

The resource name and Name tag of the load balancer.

type: string

name = "guided-lemur"

}

Genaker commented 4 years ago

I have fixed RDS issue by removing "-" sign from the name.

but this error a little bit weird :

Error: Error creating application Load Balancer: ValidationError: At least two subnets in two different Availability Zones must be specified
    status code: 400, request id: 4844ad75-4619-4565-83c5-c80c2b44faf1

  on main.tf line 1, in resource "aws_lb" "this":
   1: resource "aws_lb" "this" {
"public_subnets": {
      "value": [
        "subnet-09d599df3f68cd2f5",
        "subnet-01920b65109f712f0"
      ],
      "type": [
        "tuple",
        [
          "string",
          "string"
        ]
      ]
    },
antonbabenko commented 4 years ago

Add subnets = dependency.vpc.outputs.public_subnets into alb.

It was not specified automatically, because your ALB was not belonging to a VPC in a diagram, I suppose.

Genaker commented 4 years ago

It is belonging to the VPC... it just doesn't generate it

Can you also provide how to add these values they are missing also .

I checked old version 

security_groups = [] # @tfvars:terraform_output.security-group_5.this_security_group_id.to_list

  # A list of subnets to associate with the load balancer. e.g. ['subnet-1a2b3c4d','subnet-1a2b3c4e','subnet-1a2b3c4f']
  # type: list
  subnets = ["subnet-00703b43f38b8f971","subnet-00703b43f38b8f971","subnet-0c2dd15008a01bb51"] # @tfvars:terraform_output.vpc.public_subnets

  # VPC id where the load balancer and other resources will be deployed.
  # type: string
  vpc_id = "" # @tfvars:terraform_output.vpc.vpc_id

I have this dependencies:

dependencies { paths = ["../vpc-cloud", "../loadbalancer-internet-securitygroup"] }

  # Controls if the ALB will log requests to S3.
  # type: bool
  logging_enabled = false

  # The security groups to attach to the load balancer. e.g. ["sg-edcd9784","sg-edcd9785"]
  # type: list(string)
  security_groups = [dependency.security-group_5.outputs.this_security_group_id]

  # A list of subnets to associate with the load balancer. e.g. ['subnet-1a2b3c4d','subnet-1a2b3c4e','subnet-1a2b3c4f']
  # type: list(string)
  subnets = dependency.vpc.outputs.public_subnets

  # VPC id where the load balancer and other resources will be deployed.
  # type: string
  vpc_id = dependency.vpc.outputs.vpc_id
Genaker commented 4 years ago

it is an example - https://app.cloudcraft.co/view/f8abab63-0007-4ea2-8ea8-b9186d3259f1?key=iBtyR4AsIr9jWfy-sJARUg

it doesn't add a subnet to ELB

my blueprint is - blueprint/0b356102-1692-478a-a0fa-901c9323ab1f

Genaker commented 4 years ago

RDS name_prefix issue fixed after setting name to "mysql" without any symbols instead of "mysql-rds".

antonbabenko commented 4 years ago

A similar issue is when a security group is named ...-sg (and ...-vpc for VPC, if I remember correctly now).

Pretty weird requirements by AWS.

Genaker commented 4 years ago

After run "terragrunt apply-all" but load balancer doesn't have target groups/listeners created and so no instances are load-balanced

antonbabenko commented 4 years ago

That is because a lot of important information is not available in cloudcraft and user has to add it themself manually.

Here you can see an example of ALB module usage - https://github.com/terraform-aws-modules/terraform-aws-alb/blob/master/examples/complete-alb/main.tf

Genaker commented 4 years ago

Thanks for your replay. I will chack

On Wed, May 13, 2020 at 2:36 PM Anton Babenko notifications@github.com wrote:

That is because a lot of important information is not available in cloudcraft and user has to add it themself manually.

Here you can see an example of ALB module usage - https://github.com/terraform-aws-modules/terraform-aws-alb/blob/master/examples/complete-alb/main.tf

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/antonbabenko/modules.tf-lambda/issues/28#issuecomment-628258739, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACGJNZUXXYYOPEEH27QNUSTRRMHFBANCNFSM4LTGL5JQ .

ebahsini commented 4 years ago

That is because a lot of important information is not available in cloudcraft and user has to add it themself manually.

Here you can see an example of ALB module usage - https://github.com/terraform-aws-modules/terraform-aws-alb/blob/master/examples/complete-alb/main.tf

It doesn't appear that modules covers the use case of using named or aliased security groups, yet.

Also, as most people are designing small architectures right now, does it make sense to just create the subnets for them, baked into the terraform?

Some strategies: