hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.
https://registry.terraform.io/providers/hashicorp/aws
Mozilla Public License 2.0
9.76k stars 9.12k forks source link

[Bug]: Lakeformation tags cannot be deleted if the Glue Table was Deleted #38223

Open GabrielEisenbergOlympus opened 3 months ago

GabrielEisenbergOlympus commented 3 months ago

Terraform Core Version

1.7.5

AWS Provider Version

5.45.0

Affected Resource(s)

Expected Behavior

The tagged Glue table was deleted. Therefore, on the next terraform apply, the aws_lakeformation_resource_lf_tags resource should be removed from state as the linked resource has been destroyed.

Actual Behavior

An error was thrown in the plan which prevented an apply from occurring to delete the tags.

Relevant Error/Panic Output Snippet

Error: reading AWS Lake Formation Resource LF Tags: operation error LakeFormation: GetResourceLFTags, https response error StatusCode: 400, RequestID: fake-request-id, EntityNotFoundException: Entity Not Found

Terraform Configuration Files

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "5.45.0"
    }
  }
}

provider "aws" {
  region = "af-south-1"
}

resource "aws_glue_catalog_database" "example" {
  name = "mycatalogdatabase"
}

resource "aws_glue_catalog_table" "aws_glue_catalog_table" {
  name          = "mycatalogtable"
  database_name = aws_glue_catalog_database.example.name
}

resource "aws_lakeformation_lf_tag" "tag" {
  key    = "test"
  values = ["test"]
}

resource "aws_lakeformation_resource_lf_tags" "glue_catalogue_table_tags" {
  table {
    database_name = "mycatalogdatabase"
    name          = "mycatalogtable"
  }
  lf_tag {
    key   = "test"
    value = "test"
  }
}

Note that the database_name and name in aws_lakeformation_resource_lf_tags are purposefully stated directly so as not to have direct coupling of the resources.

Steps to Reproduce

  1. Create Glue database and table on the console. Ideally not with Terraform as we don't want to have dependencies being directly specified.
  2. Create LakeFormation tags with aws_lakeformation_lf_tag
  3. Tag the Glue table with aws_lakeformation_resource_lf_tags
  4. Delete the Glue table.
  5. Run terraform plan.

Debug Output

No response

Panic Output

No response

Important Factoids

Ordinarily, the Glue table would be created in the same configuration but in this case it is not as the creation is being done with dbt and is being handled in a separate process. The tagging is therefore being done in it's own terraform environment. This loose coupling unveiled that a deleted table is not catered for.

After a very brief look at the provider code. It seems like a missing table is not being accounted for. Go is not my forte, so please feel free to have a look and confirm this for yourself.

References

No response

Would you like to implement a fix?

None

github-actions[bot] commented 3 months ago

Community Note

Voting for Prioritization

Volunteering to Work on This Issue