(glue): Iceberg Table Support on S3Table construct

[lawofcycles]

Describe the feature

Add cfn-glue-table-opentableformatinput to the Glue S3 Table construct

In addition, create new construct TableOptimizer supporting AWS::Glue::TableOptimizer

Then add option param that create TableOptimizer corresponding to the Open Table Format based table to the Glue S3 Table construct.

Use Case

• Users will be able to define Glue Tables in CDK that support Iceberg and Open Table Format
• Users can easily create a corresponding TableOptimizer when defining Iceberg-based Glue Tables

Acknowledgements

• [X] I may be able to implement this feature request
• [ ] This feature might incur a breaking change

[lawofcycles]

I'm willing to work on implementing this feature myself if my idea looks good.

[tim-finnigan]

Thanks for creating this feature request and offering to work on a PR - I think this sounds like a good idea.

[Smotrov]

So far you can do like this to create an Iceberg table

  const myTable = new glue.S3Table(props.scope, 'IcebergTest2', {
    database: props.database,
    tableName: 'iceberg_test2',
    bucket: props.bucket,
    s3Prefix: 'iceberg_test2',
    dataFormat: glue.DataFormat.PARQUET,
    columns: [{
      name: 'col1',
      type: glue.Schema.STRING,
    }],
  });

  // Hack starts here to make the table Iceberg
  const cfnTable = myTable.node.defaultChild as mainGlue.CfnTable;

  cfnTable.openTableFormatInput = {
    icebergInput: {
      metadataOperation: 'CREATE',
      version: '2',
    }
  };

[Rizxcviii]

Just as a side note, there is currently an open issue in CloudFormation whereby you cannot update a iceberg table after creation, as the table format will be deleted when an update takes place in the CloudFormation template. Just a little thing to make note of https://github.com/aws-cloudformation/cloudformation-coverage-roadmap/issues/1919#issuecomment-2273385344