Lakeformation SELECT table permission always shows as a change

danielmitchell commented 3 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform: v0.13.16 Provider: v3.25.0

Affected Resource(s)

aws_lakeformation_permissions

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

locals {
  db_name    = "test-db"
  table_name = "test-table"
  principal  = "arn:aws:iam::367740853123:role/TestRole"
}

resource "aws_glue_catalog_database" "test" {
  name = local.db_name
}

resource "aws_glue_catalog_table" "test" {
  name          = local.table_name
  database_name = aws_glue_catalog_database.test.name
}

resource "aws_lakeformation_permissions" "test" {
  principal                     = local.principal
  permissions                   = ["ALL", "ALTER", "DELETE", "DESCRIBE", "DROP", "INSERT", "SELECT"]
  permissions_with_grant_option = ["ALL", "ALTER", "DELETE", "DESCRIBE", "DROP", "INSERT", "SELECT"]
  table {
    database_name = aws_glue_catalog_table.test.database_name
    name          = aws_glue_catalog_table.test.name
  }
}

Debug Output

Panic Output

Expected Behavior

The table permissions have not changed since the last successful apply so running a new plan should show no changes

Actual Behavior

Running a new plan always shows that the SELECT permission needs to be added. Running this apply is successful and the permissions are recreated but next time a plan is run it again shows this as a change.

This is because AWS Lakeformation treats the SELECT permission differently to the other table permissions and creates it as a column permission. This column permission is not correctly evaluated by the plan.

# module.infrastructure.aws_lakeformation_permissions.test must be replaced
-/+ resource "aws_lakeformation_permissions" "test" {
        catalog_resource              = false
      ~ id                            = "238539814" -> (known after apply)
      ~ permissions                   = [ # forces replacement
            "ALL",
            "ALTER",
            "DELETE",
            "DESCRIBE",
            "DROP",
            "INSERT",
          + "SELECT",
        ]
      ~ permissions_with_grant_option = [ # forces replacement
            "ALL",
            "ALTER",
            "DELETE",
            "DESCRIBE",
            "DROP",
            "INSERT",
          + "SELECT",
        ]

Steps to Reproduce

Run terraform apply
Run terraform apply or terraform plan

Important Factoids

References

bflad commented 3 years ago

In my recent test updates (https://github.com/hashicorp/terraform-provider-aws/pull/17316), I was able to verify that when SELECT permissions were present in the permissions and permissions with grants lists, the resource did have difference output like that noted in the report here. I did not do any research into the matter, but will mark this as a bug for future investigators to look deeper.

tuannguyen0901 commented 3 years ago

I'm facing the same errors. Furthermore, even permissions_with_grant_option = [], the state output still added ["ALL", "ALTER", "DELETE", "DESCRIBE", "DROP", "INSERT", "SELECT"]

hojatbay commented 3 years ago

I also noticed if I have something like this:

resource "aws_lakeformation_permissions" "perm1" {
  principal                     = var.principal
  permissions                   = ["SELECT"]
  table {
    database_name = var.db
    name = var.table
    wildcard = true
  }
}
resource "aws_lakeformation_permissions" "perm2" {
  principal                     = var.principal
  permissions                   = ["INSERT"]
  table {
    database_name = var.db
    name = var.table
    wildcard = true
  }
}

On terraform apply it creates two resources properly. But if I run terraform apply with no changes then it drops the perm2 resource and keeps the perm1. If I run terraform destroy with no changes then it drops the perm2 and keeps the perm1.

jackbatzner commented 3 years ago

This is intentional behavior of the API it sounds,

The SELECT permission always appears on the Data permissions page of the Lake Formation console as a separate row.

https://docs.aws.amazon.com/lake-formation/latest/dg/lf-permissions-reference.html#perm-select

I'll take a stab at getting this resolved

jackbatzner commented 3 years ago

Was able to verify this behavior with a test,

=== RUN TestAccAWSLakeFormation_serial/Permissions/selectPermissions resource_aws_lakeformation_permissions_test.go:215: Step 1/2 error: After applying this test step, the plan was not empty. stdout:

    An execution plan has been generated and is shown below.
    Resource actions are indicated with the following symbols:
    -/+ destroy and then create replacement

    Terraform will perform the following actions:

      # aws_lakeformation_permissions.test must be replaced
    -/+ resource "aws_lakeformation_permissions" "test" {
          ~ id                            = "840914301" -> (known after apply)
          ~ permissions                   = [
                # (5 unchanged elements hidden)
                "INSERT",
              + "SELECT",
            ]
          ~ permissions_with_grant_option = [
                # (5 unchanged elements hidden)
                "INSERT",
              + "SELECT",
            ]
            # (2 unchanged attributes hidden)

          + data_location {
              + arn        = (known after apply)
              + catalog_id = (known after apply)
            }

          + database {
              + catalog_id = (known after apply)
              + name       = (known after apply)
            }

          ~ table {
              ~ catalog_id    = "723063952537" -> (known after apply)
                name          = "tf-acc-test-4091274688649715014"
                # (2 unchanged attributes hidden)
            }

          + table_with_columns {
              + catalog_id            = (known after apply)
              + column_names          = (known after apply)
              + database_name         = (known after apply)
              + excluded_column_names = (known after apply)
              + name                  = (known after apply)
            }
        }

    Plan: 1 to add, 0 to change, 1 to destroy.

ghost commented 3 years ago

This has been released in version 3.33.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

ghost commented 3 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

hashicorp / terraform-provider-aws