[Enhancement]: r/aws_glue_catalog_table_optimizer Add support for expiring snapshots and orphan file deletion

[nickdelnano]

Description

AWS has released new features to Glue table optimizers to managed storage. This issue tracks adding the new features to aws_glue_catalog_table_optimizer resource.

aws_glue_catalog_table_optimizer resource was added in https://github.com/hashicorp/terraform-provider-aws/pull/38052.

Affected Resource(s) and/or Data Source(s)

aws_glue_catalog_table_optimizer

Potential Terraform Configuration

# Compaction (already exists)
resource "aws_glue_catalog_table_optimizer" "example" {
  catalog_id    = "123456789012"
  database_name = "example_database"
  table_name    = "example_table"
  configuration {
    role_arn = "arn:aws:iam::123456789012:role/example-role"
    enabled  = true
  }
  type = "compaction"
}

# Snapshot expiration (new)
resource "aws_glue_catalog_table_optimizer" "example" {
  catalog_id    = "123456789012"
  database_name = "example_database"
  table_name    = "example_table"
  configuration {
    role_arn = "arn:aws:iam::123456789012:role/example-role"
    enabled  = true
    retentionConfiguration {
      icebergConfiguration {
        snapshotRetentionPeriodInDays = 1
        numberOfSnapshotsToRetain = 2
        cleanExpiredFiles = true
      }
    }
  }
  type = "retention"
}

# Orphan file removal (new)
resource "aws_glue_catalog_table_optimizer" "example" {
  catalog_id    = "123456789012"
  database_name = "example_database"
  table_name    = "example_table"
  configuration {
    role_arn = "arn:aws:iam::123456789012:role/example-role"
    enabled  = true
    orphanFileDeletionConfiguration {
      icebergConfiguration {
        orphanFileRetentionPeriodInDays = 5
        location = "s3://example-bucket/path"
      }
    }
  }
  type = "orphan_file_deletion"
}

References

CreateTableOptimizer API docs https://docs.aws.amazon.com/glue/latest/webapi/API_CreateTableOptimizer.html UpdateTableOptimizer API docs https://docs.aws.amazon.com/glue/latest/webapi/API_UpdateTableOptimizer.html

The individual API docs aren't updated yet but the Table Optimizer API docs are. Summary of changes:

• TableOptimizer - type field now supports values retention and orphan_file_deletion
• TableOptimizerConfiguration - expanded to support more fields
• RetentionConfiguration - new
• IcebergRetentionConfiguration - new
• OrphanFileDeletionConfiguration - new
• IcebergOrphanFileDeletionConfiguration - new

Would you like to implement a fix?

None

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[ryanmiville]

I am interested in working on this issue.

[nickdelnano]

@ryanmiville please go ahead!