aws_lakeformation_permissions can't be created on lf_tags shared across accounts

[Fanswers]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v1.2.8 on windows_amd64

• provider registry.terraform.io/hashicorp/aws v4.27.0
• provider registry.terraform.io/hashicorp/local v2.2.3

Affected Resource(s)

• aws_lakeformation_permissions

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

template file :

variable "lf_tag_permissions" {
  type = object({
    permissions           = list(string) # ASSOCIATE, DESCRIBE
    grantable_permissions = list(string) # ASSOCIATE, DESCRIBE
  })
  description = <<EOT
Object containing the permissions to be granted to the LF-tags.
Keys: permissions and grantable_permissions
Possible values for permissions and grantable_permissions: ASSOCIATE, DESCRIBE or empty list

Format :
{
  permissions           = ["DESCRIBE"]
  grantable_permissions = []
}
EOT

  nullable = true
  default  = null
}

variable "database_permissions" {
  type = object({
    permissions           = list(string) # ALL, ALTER, CREATE_TABLE, DESCRIBE and DROP
    grantable_permissions = list(string) # ALL, ALTER, CREATE_TABLE, DESCRIBE and DROP
  })
  description = <<EOT
Object containing the permissions to be granted to the Database.
Keys: permissions and grantable_permissions
Possible values for permissions and grantable_permissions: ALL, ALTER, CREATE_TABLE, DESCRIBE, DROP or empty list

Format :
{
  permissions           = ["ALTER", "DESCRIBE"]
  grantable_permissions = ["DESCRIBE"]
}
EOT

  nullable = true
  default  = null
}

variable "table_permissions" {
  type = object({
    permissions           = list(string) # ALL, ALTER, DELETE, DESCRIBE, DROP, INSERT and SELECT
    grantable_permissions = list(string) # ALL, ALTER, DELETE, DESCRIBE, DROP, INSERT and SELECT
  })
  description = <<EOT
Object containing the permissions to be granted to the Database.
Keys: permissions and grantable_permissions
Possible values for permissions and grantable_permissions: ALL, ALTER, DELETE, DESCRIBE, DROP, INSERT, SELECT or empty list

Format :
{
  permissions           = ["DESCRIBE", "INSERT", "SELECT"]
  grantable_permissions = ["DESCRIBE", "SELECT"]
}
EOT

  nullable = true
  default  = null
}

variable "lf_tags" {
  type        = any
  description = <<EOT
Map containing LF-tags to assign to resources.
/!\ You must specify a unique LF-tag key. Otherwise, the last value will be used.

Format:
{
  lf-tag_key : ["lf-tag_value1", "lf-tag_value2"],
  lf-tag_key_2: ["lf-tag_value3"]
}
EOT

  nullable = false
}

variable "auto_cross_account_permissions" {
  type        = bool
  description = "Boolean used to set the permissions of LF-tags, database and table needed to use the cross-account. Only works if no other permission is implemented in the module."

  nullable = false
  default  = false
}

variable "iam_principals" {
  type        = list(string)
  description = "Add a list of one or more IAM users or roles."

  nullable = true
  default  = []
}

variable "saml_arn" {
  type        = list(string)
  description = "Enter a list of one or more SAML user or group ARNs or Amazon QuickSight ARN."

  nullable = true
  default  = []
}

variable "external_accounts" {
  type        = list(string)
  description = "Enter a list of one or more AWS account IDs or AWS organization IDs."

  nullable = true
  default  = []
}

##### Manage whether the user wants to use the automatic permissions for cross-accounting or not. #####
locals {

  lf_tag_permissions = var.auto_cross_account_permissions == true ? {
    permissions           = ["DESCRIBE"]
    grantable_permissions = []
  } : var.lf_tag_permissions

  database_permissions = var.auto_cross_account_permissions == true ? {
    permissions           = ["DESCRIBE"]
    grantable_permissions = ["DESCRIBE"]
  } : var.database_permissions

  table_permissions = var.auto_cross_account_permissions == true ? {
    permissions           = ["DESCRIBE", "SELECT"]
    grantable_permissions = ["DESCRIBE", "SELECT"]
  } : var.table_permissions
}

##### Manage possible permission values for LF-tags, databases and tables #####
locals {
  accepted_lf_tag_permissions   = ["ASSOCIATE", "DESCRIBE"]
  accepted_database_permissions = ["ALL", "ALTER", "CREATE_TABLE", "DESCRIBE", "DROP"]
  accepted_table_permissions    = ["ALL", "ALTER", "DELETE", "DESCRIBE", "DROP", "INSERT", "SELECT"]
}

resource "aws_lakeformation_permissions" "create_database_permission" {
  for_each = local.database_permissions != null ? toset(concat(var.iam_principals, var.saml_arn, var.external_accounts)) : []

  lifecycle {

    precondition {
      # Here we use the value of the variable and not the local value to be sure that the user has not asked for automatic permissions and has set specific conditions.
      condition     = !(var.auto_cross_account_permissions == true && var.database_permissions != null)
      error_message = "Error! If you want to use automatic values for cross-accounting, remove all other permission implementations (`lf_tag_permissions`, `database_permissions`, `table_permissions`)"
    }

    precondition {
      condition     = alltrue([for permission in local.database_permissions.permissions : contains(local.accepted_database_permissions, permission)])
      error_message = "Error! Database permissions possible values: ALL, ALTER, CREATE_TABLE, DESCRIBE or DROP"
    }

    precondition {
      condition     = alltrue([for grantable_permission in local.database_permissions.grantable_permissions : contains(local.accepted_database_permissions, grantable_permission)])
      error_message = "Error! Database grantable permissions possible values: ALL, ALTER, CREATE_TABLE, DESCRIBE or DROP"
    }
  }

  principal                     = each.value
  permissions                   = sort(local.database_permissions.permissions)
  permissions_with_grant_option = local.database_permissions.grantable_permissions == null ? [] : sort(local.database_permissions.grantable_permissions)

  lf_tag_policy {
    resource_type = "DATABASE"

    dynamic "expression" {
      for_each = var.lf_tags
      content {
        key    = expression.key
        values = expression.value
      }
    }
  }
}

resource "aws_lakeformation_permissions" "create_table_permission" {
  for_each = local.table_permissions != null ? toset(concat(var.iam_principals, var.saml_arn, var.external_accounts)) : []

  lifecycle {
    precondition {
      # Here we use the value of the variable and not the local value to be sure that the user has not asked for automatic permissions and has set specific conditions.
      condition     = !(var.auto_cross_account_permissions == true && var.table_permissions != null)
      error_message = "Error! If you want to use automatic values for cross-accounting, remove all other permission implementations (`lf_tag_permissions`, `database_permissions`, `table_permissions`)"
    }

    precondition {
      condition     = alltrue([for permission in local.table_permissions.permissions : contains(local.accepted_table_permissions, permission)])
      error_message = "Error! Table permissions possible values: ALL, ALTER, DELETE, DESCRIBE, DROP, INSERT or SELECT"
    }

    precondition {
      condition     = alltrue([for grantable_permission in local.table_permissions.grantable_permissions : contains(local.accepted_table_permissions, grantable_permission)])
      error_message = "Error! Table grantable permissions possible values: ALL, ALTER, DELETE, DESCRIBE, DROP, INSERT or SELECT"
    }
  }

  principal                     = each.value
  permissions                   = sort(local.table_permissions.permissions)
  permissions_with_grant_option = local.table_permissions.grantable_permissions == null ? [] : sort(local.table_permissions.grantable_permissions)

  lf_tag_policy {
    resource_type = "TABLE"

    dynamic "expression" {
      for_each = var.lf_tags
      content {
        key    = expression.key
        values = expression.value
      }
    }
  }
}

main.tf :

module "datalake_permissions_on_lf_tag" {
  source = "../../../../../modules/lakeformation-lf-tag-permission"

  iam_principals = ["arn:aws:iam::000011112222:role/account-name"]
  lf_tags = { "example" : ["VALUE1"] }

  database_permissions = {
    permissions           = ["DESCRIBE"]
    grantable_permissions = []
  }

  table_permissions = {
    permissions           = ["DESCRIBE", "SELECT"]
    grantable_permissions = []
  }
}

Expected Behavior

An AWS Lake Formation Permissions should be created on the lf tag who's shared between differents accounts.

Actual Behavior

Terraform can't find the lf tag targeted, it raise an EntityNotFoundException error. If we precise the catalog_id of the lf tag it will raise another error saying : "error reading Lake Formation permissions: timeout while waiting for state to become 'AVAILABLE' (last state: 'NOT FOUND', timeout: 1m0s)"
It is however possible to do it by hand an with the AWS CLI.

Steps to Reproduce

1. terraform plan
2. terraform apply

[Witekkq]

The issue is not related to terraform directly, the TF generates the query correctly but AWS response is [] for resource type table even for cli direct calls. { "Principal": { "DataLakePrincipalIdentifier": "arn:aws:iam::xxx" }, "Resource": { "LFTagPolicy": { "CatalogId": "central_account_id", "ResourceType": "TABLE", "Expression": [ { "TagKey": "lob", "TagValues": [ "line" ] }, { "TagKey": "lob.line", "TagValues": [ "lambo" ] } ] } } }

-> aws lakeformation list-permissions --cli-input-json file://test_above.json { "PrincipalResourcePermissions": [] } and it should find something in my configuration, for type "DATABASE" it's ok.

[Fanswers]

Hi, Thank's for your answer, but I don't think it respond to my problem. Maybe I've miss understand something.

The error is that Terraform can't create datalake permissions on lf_tags shared across accounts. But we can do it with the grant-permissions query on the AWS CLI. So I think the problem come from terraform.

[Witekkq]

It's not a solution, just a note left here, that the issue is probably in different repository, as AWS api call return value is wrong, and do not include ""SELECT" values so terraform cannot confirm that resource is being created.

[peter-resnick]

We are also experiencing this issue - anyone able to find a solve for this?

[Fanswers]

We are also experiencing this issue - anyone able to find a solve for this?

In waiting that this issue is solve, we used the AWS CLI via terraform to create our lf tags. It's not a very clean solution but this is the only one I've found to skirt this bug.

[bruj0]

We have also being hit by this bug. Terraforms produces this error after the apply but the permission is actually created:

Error: error reading Lake Formation permissions: timeout while waiting for state to become 'AVAILABLE' (last state: 'NOT FOUND', timeout: 1m0s)

Here is the reproduction steps using AWS CLI:

{
    "CatalogId": "local_catalog",
    "Permissions": [
        "DESCRIBE"
    ],
    "Principal": {
        "DataLakePrincipalIdentifier": "arn:aws:iam::local_catalog:role/myrole"
    },
    "Resource": {
        "LFTagPolicy": {
            "CatalogId": "remote_catalog",
            "Expression": [
                {
                    "TagKey": "data_layer",
                    "TagValues": [
                        "gold"
                    ]
                }
            ],
            "ResourceType": "DATABASE"
        }
    }
}

$ aws lakeformation grant-permissions   --cli-input-json file://test-create.json

{
    "CatalogId": "local_catalog",
    "Principal": {
        "DataLakePrincipalIdentifier": "arn:aws:iam::local_catalog:role/myrole"
    },
    "Resource": {
        "LFTagPolicy": {
            "CatalogId": "remote_catalog",
            "Expression": [
                {
                    "TagKey": "data_layer",
                    "TagValues": [
                        "gold"
                    ]
                }
            ],
            "ResourceType": "DATABASE"
        }
    }
}

$ aws lakeformation list-permissions   --cli-input-json file://test-list.json
{
    "PrincipalResourcePermissions": []
}

[lulgithub]

Does anyone know if this bug is resolved or not?

[Fanswers]

Does anyone know if this bug is resolved or not?

Doesn't seems resolved no 😕

[Rezkmike]

the issue still open

[lulgithub]

Thank you!

[dacreify]

We encountered this issue as well but discovered a work-around, at least for our setup. If we only specify some of the tags and values used for the cross-account sharing we get the timeout reliably, but if we specify the exact same tags and values the aws_lakeformation_permissions resource creates successfully.

I'm not sure why this works but FWIW.

[dacreify]

Of course this work-around has limited utility because you can't specify more narrow permissions to a local principal than what was granted at the account-level. Can confirm provider create narrower permissions and they work, it just times out trying to read them back.