Victor-Agali commented 1 year ago

Terraform Core Version

Terraform v1.4.2 on darwin_arm6

AWS Provider Version

4.60.0

Affected Resource(s)

Hi guys,

I am trying to deploy AWS Glue crawler with a role and two attached policies and glue database. While deploying it deploys the the role and policies only for the first time and it fails. During the second attempt, it deploys the crawler successfully. It should deploy all the resources ideally.

Please find the terraform below iam.tf

resource aws_glue_crawler glue_crawler {
  database_name =  "database_name"
  name          =  "aws-crawler"
  role          =  aws_iam_role.iam_role.name
  configuration = jsonencode(
    {
      "Version"       = 1.0,
      "CrawlerOutput"   = {
        "Partitions"  = { "AddOrUpdateBehavior" = "InheritFromTable"}
      },

     Grouping          =     { TableLevelConfiguration = 6 }
    }
  )

  schema_change_policy {
    update_behavior   = var.update_behaviour
    delete_behavior   = var.delete_behaviour
  }

  s3_target {
    path = "s3://test-bucket"
  }

  recrawl_policy {
    recrawl_behavior  = var.recrawl_behaviour
  }
}

provider "aws" {
  region = "eu-west-1"
}

terraform {
  backend "s3" {
    aws = {
      source = "hashicorp/aws"
      version = "= 4.60.0"
    }
  }
  required_version = ">=1.0.0"
}

IAM - Role

resource aws_iam_role iam_role {
  name               = "${var.environment}-iam-role"
  assume_role_policy = templatefile("templates/iam_role.json", {
    environment      = var.environment
  })
}

resource aws_iam_policy policy {
  name = "${var.environment}-iam-policy"
  policy = templatefile("templates/iam_policy.json", {
    environment   : var.environment
  })
}

# Lambda IAM - Policy Attachment
resource aws_iam_role_policy_attachment iam_policy_attachment_1 {
  role              = aws_iam_role.iam_role.name
  policy_arn        = aws_iam_policy.policy.arn
}

# Lambda IAM - Policy Attachment
resource aws_iam_role_policy_attachment iam_policy_attachment_2 {
  role              = aws_iam_role.iam_role.name
  policy_arn        = "arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole"
}
main.tf

module iam {
  source         = "./modules/iam"
  environment    = var.environment
  aws_account_id = local.aws_account_id
  s3_location    = var.s3_location
  kms_key        = data.aws_kms_key.lakehouse_landing_zone_key.arn
  glue_iam_role  = module.iam.glue_iam_role
}

module glue_database {
  source         = "./modules/database"
  environment    = var.environment
  glue_database  = var.glue_database_name
}

module aws_glue_crawler {
  source         = "./modules/iam"
  crawler_name   = "aws-crawler"
  database_name  = "module.glue_database.glue_database"
  environment    = var.environment
  glue_iam_role  = module.iam.glue_iam_role
  glue_iam_policy = module.iam.policy
  s3_location    = var.s3_location
  s3_target      = {
    path = "s3://test-bucket"
  }
}

Expected Behavior

Apply complete! Resources: 6 added, 0 changed, 0 destroyed

Actual Behavior

Actual behaviour Failed with error

Error: creating Glue Crawler (aws-crawler): InvalidInputException: Service is unable to assume provided role. Please verify role's TrustPolicy
│ 
│   with module.aws_glue_crawler.aws_glue_crawler.glue_crawler,
│   on .terraform/modules/aws_glue_crawler/modules/iam.tf line 1, in resource "aws_glue_crawler" "glue_crawler":
│    1: resource aws_glue_crawler glue_crawler {
│ 
╵

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

*

Steps to Reproduce

Terraform apply

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

None

github-actions[bot] commented 1 year ago

Community Note

Voting for Prioritization

Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
Please see our prioritization guide for information on how we prioritize.
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

If you are interested in working on this issue, please leave a comment.
If this would be your first contribution, please review the contribution guide.

mattburgess commented 1 year ago

Hi @Victor-Agali. This problem is likely due to the role's policy attachments not happening before the Glue Crawler creation is attempted. This is quite a familiar problem/pattern across a number of AWS services. You will likely find adding explicit dependencies on those resources will fix the ordering issue. e.g.

resource aws_glue_crawler glue_crawler {
...

    depends_on = [aws_iam_role_policy_attachment.iam_policy_attachment_1, aws_iam_role_policy_attachment.iam_policy_attachment_2]
}

Victor-Agali commented 1 year ago

@mattburgess so i am deploying both resources from 2 different repositories, I am outputting the role and policy arns as follows

outputs.tf (iam)

Output policy_attachment_1{
value = aws_iam_role_policy_attachment.iam_policy_attachment_1
}

Output policy_attachment_2{
value = aws_iam_role_policy_attachment.iam_policy_attachment_2
}

so would adding it as variables.tf

variable policy_1 {
type = string
default ="The first policy to be attached"}

variable policy_2 {
type = string
default ="The second policy to be attached"}

main.tf

module iam {
source = "./modules/iam"
policy_1 = module.iam.policy_attachment_1
policy_2 = module.iam.policy_attachment_2
}

module crawler{
source = "./modules/glue_crawler"
depends_on [module.iam.policy_1, module.iam.policy_2]
}

or do i have to add the depends_on to where the resource is created ( resource aws_glue_crawler glue_crawler { ...

mattburgess commented 1 year ago

I'd be tempted to add them into the resource aws_glue_crawler glue_crawler{} as per my previous suggestion. I think this might work:

resource aws_glue_crawler glue_crawler {
...

    depends_on = [module.iam.aws_iam_role_policy_attachment.iam_policy_attachment_1, module.iam.aws_iam_role_policy_attachment.iam_policy_attachment_2]
}

Note that the above doesn't use the module's outputs; the dependencies are on the resources themselves, as stored in the state, rather than any specific attributes (e.g. arn) of them.

Victor-Agali commented 1 year ago

@mattburgess Unfortunately this does not still work and the same error comes up, desperately need it to work as i would need to run it in a Jenkins pipeline. i set it up as follows

iam.tf

resource aws_iam_role iam_role {
  name               = "${var.environment}-iam-role"
  assume_role_policy = templatefile("templates/iam_role.json", {
    environment      = var.environment
  })
}

resource aws_iam_policy policy {
  name = "${var.environment}-iam-policy"
  policy = templatefile("templates/iam_policy.json", {
    environment   : var.environment
  })
}

# Lambda IAM - Policy Attachment
resource aws_iam_role_policy_attachment iam_policy_attachment_1 {
  role              = aws_iam_role.iam_role.name
  policy_arn        = aws_iam_policy.policy.arn
}

# Lambda IAM - Policy Attachment
resource aws_iam_role_policy_attachment iam_policy_attachment_2 {
  role              = aws_iam_role.iam_role.name
  policy_arn        = "arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole"
}

resource aws_glue_crawler glue_crawler {
  depends_on = [aws_iam_role_policy_attachment.iam_policy_attachment_1,aws_iam_role_policy_attachment.iam_policy_attachment_2]
  database_name = var.database_name
  name          = var.crawler_name
  role          = var.glue_iam_role
  configuration = jsonencode(
    {
      "Version"       = 1.0,
      "CrawlerOutput"   = {
        "Partitions"  = { "AddOrUpdateBehavior" = "InheritFromTable"}
      },

      Grouping          =     { TableLevelConfiguration = 6 }
    }
  )

  schema_change_policy {
    update_behavior   = var.update_behaviour
    delete_behavior   = var.delete_behaviour
  }

  s3_target {
    path = "s3://${var.environment}-${var.s3_location}"
  }

  recrawl_policy {
    recrawl_behavior  = var.recrawl_behaviour
  }
}

main.tf

module iam {
  source         = "./modules/iam"
  environment    = var.environment
  aws_account_id = local.aws_account_id
  s3_location    = var.s3_location
  kms_key        = data.aws_kms_key.lakehouse_landing_zone_key.arn
  glue_iam_role  = module.iam.glue_iam_role
  crawler_name   = "${var.environment}-${var.glue_crawler_name}"
  database_name  = "${var.environment}-${var.glue_database_name}"
}

bigwheel commented 1 year ago

I think this is caused by https://github.com/hashicorp/terraform-provider-aws/pull/30371 and it fixed in https://github.com/hashicorp/terraform-provider-aws/pull/30370 .

Victor-Agali commented 1 year ago

@bschaatsbergen hi could you look into this please? my crawler still fails to create on first apply and i dont understand how #30370 fixes it

bschaatsbergen commented 1 year ago

Hi @Victor-Agali this fix should be released soon in the 4.62.0 provider version.

gdavison commented 1 year ago

@Victor-Agali, did v4.62.0 solve the problem for you or are you still seeing it?

hashicorp / terraform-provider-aws

terraform apply not deploying all the resources specified at the same time #30389

Terraform Core Version

AWS Provider Version

Affected Resource(s)

Expected Behavior

Actual Behavior

Relevant Error/Panic Output Snippet

Terraform Configuration Files

Steps to Reproduce

Debug Output

Panic Output

Important Factoids

References

Would you like to implement a fix?

Community Note