resource awscc_lakeformation_data_cells_filter argument column_wildcard Bug

Liangtian-hcp commented 2 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment
The resources and data sources in this provider are generated from the CloudFormation schema, so they can only support the actions that the underlying schema supports. For this reason submitted bugs should be limited to defects in the generation and runtime code of the provider. Customizing behavior of the resource, or noting a gap in behavior are not valid bugs and should be submitted as enhancements to AWS via the CloudFormation Open Coverage Roadmap.

Terraform CLI and Terraform AWS Cloud Control Provider Version

Affected Resource(s)

*awscc_lakeformation_data_cells_filter

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "awscc_lakeformation_data_cells_filter" "emp-edu-filter-all" {
  name             = "emp-edu-filter-all"
  database_name    = "common_test"
  table_name       = "master_emp_education"
  table_catalog_id = "123456789012"
  column_wildcard = {}
  row_filter = {
    filter_expression = "person_id !=''"
  }
}

Debug Output

Panic Output

Expected Behavior

create resource emp-edu-filter-all

Actual Behavior

return Error: AWS SDK Go Service Operation Incomplete with awscc_lakeformation_data_cells_filter.emp-edu-filter-all on test-mdm-common-lakeformation.tf line 114, in resource "awscc_lakeformation_data_cells_filter" "emp-edu-filter-all": resource "awscc_lakeformation_data_cells_filter" "emp-edu-filter-all" { Waiting for Cloud Control API service CreateResource operation completion returned: waiter state transitioned to FAILED. StatusMessage: Data cells filter must have column names or column wildcard (Service: LakeFormation, Status Code: 400, Request ID: 8e6279a4-5b51-4330-a6a0-12312312). ErrorCode: InvalidRequest

Steps to Reproduce

terraform apply

Important Factoids

I checked the cloudtrail logs ，I got this error message：

"eventTime": "2022-09-06T03:08:55Z",
    "eventSource": "lakeformation.amazonaws.com",
    "eventName": "CreateDataCellsFilter",
    "awsRegion": "ap-southeast-1",
    "sourceIPAddress": "cloudformation.amazonaws.com",
    "userAgent": "cloudformation.amazonaws.com",
    "errorCode": "InvalidInputException",
    "errorMessage": "Data cells filter must have column names or column wildcard",
    "requestParameters": {
        "tableData": {
            "tableCatalogId": 123456789012",
            "databaseName": "common_test",
            "tableName": "master_emp_education",
            "name": "emp-edu-filter-all"
        }
    },
    "responseElements": null,
    "requestID": "8e6279a4-5b51-4330-a6a0-12314124134",
    "eventID": "6f144869-4d89-4e48-9f7a-12312312312",
It's looks like awscc 0.31.0 remove the "column_wildcard = {}" argument.

References

0000

ewbankkit commented 2 years ago

We do not currently send empty objects over to Cloud Control: https://github.com/hashicorp/terraform-provider-awscc/blob/8fc6356e37ce21e2a4acd1d4576d7017ded5e23a/internal/generic/translate.go#L168-L170.

That functionality has been there from the start. There may be issues with diffs if we remove this.

ericodell commented 1 year ago

This seems to be a persistent problem if users want to use column_wildcard with no excluded columns. ie specifying:

"column_wildcard": {},

fails with:

│ Waiting for Cloud Control API service CreateResource operation completion │ returned: waiter state transitioned to FAILED. StatusMessage: Data cells │ filter must have column names or column wildcard

because tf awscc fails to include the column_wildcard argument.

hpejman commented 1 year ago

We are also facing a similar failures when using awscc provider. In our use case, we need to create data cells filters to exclude certain rows. However, we need all columns in the rows that qualify for the filter expression to be included. We can achieve this by using aws lakeformation create-data-cells-filter command and passing it the following: { "TableCatalogId": "\<aws account>", "DatabaseName": "\<database name>", "TableName": "\<table name>", "Name": "\<data filter name>", "RowFilter": { "FilterExpression": "\<expression>" }, "ColumnNames": [], "ColumnWildcard": { "ExcludedColumnNames": [] } }

However, we are not able to achieve the same results using awscc provider due to the following error: Waiting for Cloud Control API service CreateResource operation completion returned: waiter state transitioned to FAILED. StatusMessage: Data cells filter must have column names or column wildcard

hashicorp / terraform-provider-awscc