Bug: Deletion of LakeFormation Permissions breaks Glue Catalog Database Deletion

[thenaturalist]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Affected Resource(s)

• aws_glue_catalog_database
• aws_lakeformation_permissions

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

Vanilla aws provider with S3 remote backend

Debug Output

Panic Output

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:

  # aws_glue_catalog_database.foobar_database will be destroyed
  # (because aws_glue_catalog_database.foobar_database is not in configuration)
  - resource "aws_glue_catalog_database" "foobar" {
      - arn        = "arn:aws:glue:FOO-REGION:FOO-ACCOUNT-ID:database/foobar" -> null
      - catalog_id = "FOO-ACCOUNT-ID" -> null
      - id         = "FOO-ACCOUNT-ID:foobar" -> null
      - name       = "foobar" -> null
      - parameters = {} -> null
    }

  # aws_lakeformation_permissions.foo_all_admin will be destroyed
  # (because aws_lakeformation_permissions.foo_all_admin is not in configuration)
  - resource "aws_lakeformation_permissions" "foo_all_admin" {
      - catalog_resource              = false -> null
      - id                            = "12345" -> null
      - permissions                   = [
          - "ALL",
          - "ALTER",
          - "CREATE_TABLE",
          - "DESCRIBE",
          - "DROP",
        ] -> null
      - permissions_with_grant_option = [
          - "ALL",
          - "ALTER",
          - "CREATE_TABLE",
          - "DESCRIBE",
          - "DROP",
        ] -> null
      - principal                     = "arn:aws:iam::FOO-ACCOUNT-ID:role/FOO-ROLE" -> null

      - database {
          - catalog_id = "FOO-ACCOUNT-ID" -> null
          - name       = "foobar" -> null
        }
    }

Plan: 0 to add, 0 to change, 2 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_lakeformation_permissions.foo_all_admin: Destroying... [id=12345]
aws_lakeformation_permissions.foo_all_admin: Destruction complete after 1s
aws_glue_catalog_database.foobar_database: Destroying... [id=FOO-ACCOUNT-ID:foobar]
╷
│ Error: Error deleting Glue Catalog Database: AccessDeniedException: Insufficient Lake Formation permission(s): Required Drop on foobar
│ 
│ 
╵

Expected Behavior

The database is correctly deleted.

Actual Behavior

The Lake Formation permission granting the provider role ALL permissions on the glue database is deleted prior to the database itself. The above error is produced and the apply fails.

To circumvent this behavior, the permissions need to be re-added and both permissions and database deleted in conjunction again. Typically this works only on the 2nd or 3rd try.

Steps to Reproduce

1. Create a glue database
2. Create a Lakeformation Permission granting the provider IAM role or user ALL privileges on the database in 1.
3. Destroy both resources simultaneously.
4. Repeat 2-3 times as the behavior is nondeterministic and might work/ fail in different runs.

Important Factoids

References

• 0000

[justinretzolk]

Hey @thenaturalist 👋 Can you supply the Terraform configuration as well? Seeing that might help look into this a bit more. I'm particularly interested in any dependencies -- implicit or explicit -- that might exist in the config.

[GeorgeTerry]

I have a similar problem but with a Glue Table

│ Error: Error deleting Glue Catalog Table: AccessDeniedException: Insufficient Lake Formation permission(s): Required Drop on glue_table_example

[justinretzolk]

@GeorgeTerry Are you able to supply a sample configuration so that we may attempt to reproduce?

[aitanajimenez-facephi]

Hello! I have the same bug. Is there new information about it?

One database failed to delete The following database failed to delete: "xxxxx" Here is the most recent error message: Insufficient Lake Formation permission(s): Required Drop on xxxxx

[pgupta-adastra]

I have the same issue , i tried to add dependency in "aws_lakeformation_permissions" for database so it will delete database before removing permissions but this is not working.