Apache XTable (incubating) is a cross-table converter for lakehouse table formats that facilitates interoperability across data processing systems and query engines.
It is expected to capture the comments and sync the comments of field when syncing schema. The PR makes the comments of schema fields are captured and synced correctly.
Brief change log
For Delta target, extract the comment from internal schema and add to Spark schema representative.
For Iceberg target, make sure any difference of comments is captured during syncSchema() and the schema is updated with new comments.
Verify this pull request
This change added tests and can be verified as follows:
Added unit tests to test the comments in Iceberg schema sync test class
Added comments in Delta schema extractor test class
Important Read
Aim to address https://github.com/apache/incubator-xtable/issues/525
What is the purpose of the pull request
It is expected to capture the comments and sync the comments of field when syncing schema. The PR makes the comments of schema fields are captured and synced correctly.
Brief change log
Verify this pull request
This change added tests and can be verified as follows: