[Bug]: Iceberg tables schema can't be updated with Terraform

[leonardo-okada-darwin]

Terraform Core Version

1.4.0

AWS Provider Version

5.43.0

Affected Resource(s)

aws_glue_catalog_table

Expected Behavior

Iceberg tables schema updated according to new specified configuration.

Actual Behavior

Iceberg tables are updated in Glue catalog, but it's metadata stays the same, ignoring any changes in the new table configuration.

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

resource "aws_glue_catalog_table" "test_table" {
  name          = "iceberg_test"
  database_name = "datalake_stage_03_feature"

  table_type = "EXTERNAL_TABLE"

  open_table_format_input {
    iceberg_input {
      metadata_operation = "CREATE"
      version            = "2"
    }
  }

  storage_descriptor {
    location = "s3://XXX/iceberg_test/datalake_stage_03_feature.db/iceberg_test/"
    columns {
      name = "new_column_1"
      type = "string"
    }

  }
}

Steps to Reproduce

1. Create table iceberg_table with the mentioned Terraform configuration

2. Modify Terraform configuration, adding a new column named new_column_2, with any typing

3. Apply the new table configuration with Terraform

4. The new table version should have column new_column_2 in Glue Catalog, but it shouldn't be accessible in Athena as it's metadata wasn't updated

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

I believe this behaviour is caused by #33374, as it was created to keep the parameter metadata_location when updating Iceberg tables. This change indeed fixed a issue where parameters were lost when updating the table but it also meant that they would be kept static even when metadata should be changed.

Would you like to implement a fix?

None

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[jrbracey]

Is there any update on this fix?