Error: error reading Lake Formation permissions: multiple permissions found

ghost commented 3 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Provider: 3.22.0

Affected Resource(s)

aws_lakeformation_permissions

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "aws_lakeformation_permissions" "perms" {
  provider = aws
  permissions = [
    "SELECT"
  ]
  principal = data.aws_iam_role.data_analyst.arn
  table {
    database_name = "db"
    name = "table"
  }
}

Debug Output

Error: error reading Lake Formation permissions: multiple permissions found

Expected Behavior

When applying SELECT permissions to a Table resource on AWS terraform will apply permissions and exit with the above error message.

I believe this is because granting SELECT on a table resource actually provides select on * columns in that table. And when terraform tries to read back a list of permissions it fails the check found here https://github.com/hashicorp/terraform-provider-aws/blob/37e48187e9517d09c4c9bbce9e4210e8abdc5b28/aws/resource_aws_lakeformation_permissions.go#L302

Perhaps table_with_columns is a work-around but according to the source code/docs this doesn't have a Wildcard mapping to the ColumnWildcard key found in the AWS API making it difficult to use


{
    "PrincipalResourcePermissions": [
        {
            "Principal": {
                "DataLakePrincipalIdentifier": "arn:aws:iam:::role/"
            },
            "Resource": {
                "TableWithColumns": {
                    "CatalogId": "",
                    "DatabaseName": "",
                    "Name": "",
                    "ColumnWildcard": {} <---- Missing in terraform?
                }
            },
            "Permissions": [
                "SELECT"
            ],
            "PermissionsWithGrantOption": []
        }
    ]
}

Actual Behavior

When applying SELECT permissions to table resource, permissions seem to be applied but the job will fail complaining about too many permissions being found.

Steps to Reproduce

terraform apply

Important Factoids

We are running with IAM_ALLOWED_PRINCIPAL on. This also causes the above error due to an implicit Super being provided to roles that have IAM access which seems to trip the check found here https://github.com/hashicorp/terraform-provider-aws/blob/37e48187e9517d09c4c9bbce9e4210e8abdc5b28/aws/resource_aws_lakeformation_permissions.go#L302

References

0000

YakDriver commented 3 years ago

Thanks for reporting this issue! I think I see why this is happening and it's possible it's an easy fix. However, if you can provide a complete configuration with the table and roles involved, it will help to make sure we cover your specific issue.

ghost commented 3 years ago

The table is not managed by terraform so I have no configuration for that. It happens with two completely different roles I've tried.

resource "aws_iam_role" "role" {
  count = local.account_role_enabled ? 1 : 0

  name               = var.name
  assume_role_policy = data.aws_iam_policy_document.assume_role.json

  permissions_boundary = var.boundary_policy_arn
  max_session_duration = var.max_session_timeout
}

The role is just a basic role with a few attached policies, it's a pretty simple set up.

YakDriver commented 3 years ago

TL;DR: I have replicated this issue. This is a bug in the aws_lakeformation_permissions resource. We are working on a fix. I cannot provide an estimate for the fix but please know that we are actively working the issue.

For background, this problem arises because the List action can return multiple PrincipalResourcePermissions for a principal. This happens with column permissions in particular because the List action does not allow filtering results by the specific TableWithColumns but instead filters on the Table. This causes the List action to return PrincipalResourcePermissions for the Table and TableWithColumns.

An example of PrincipleResourcePermissions returned for a TableWithColumns where the principal also has permissions for the Table:

[{
  Permissions: [
    "ALL",
    "ALTER",
    "DELETE",
    "DESCRIBE",
    "DROP",
    "INSERT"
  ],
  PermissionsWithGrantOption: [
    "ALL",
    "ALTER",
    "DELETE",
    "DESCRIBE",
    "DROP",
    "INSERT"
  ],
  Principal: {
    DataLakePrincipalIdentifier: "arn:aws:iam::123456789101:user/weeknd"
  },
  Resource: {
    Table: {
      CatalogId: "123456789101",
      DatabaseName: "tf-acc-test-4640723771648502964",
      Name: "tf-acc-test-4640723771648502964"
    }
  }
}, {
  Permissions: ["SELECT"],
  PermissionsWithGrantOption: ["SELECT"],
  Principal: {
    DataLakePrincipalIdentifier: "arn:aws:iam::123456789101:user/weeknd"
  },
  Resource: {
    TableWithColumns: {
      CatalogId: "123456789101",
      ColumnWildcard: {

      },
      DatabaseName: "tf-acc-test-4640723771648502964",
      Name: "tf-acc-test-4640723771648502964"
    }
  }
}]

YakDriver commented 3 years ago

Thank you for reporting this issue! A fix has been merged. As a new service with an uncommon API, we appreciate your continued support in helping to mature Lake Formation in the AWS provider!

ghost commented 3 years ago

This has been released in version 3.25.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

ghost commented 3 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

hashicorp / terraform-provider-aws