An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
This PR builds on the base changes for introducing DomainMetadata into the Kernel, which are not yet merged. For changes specific to this PR, please refer to the last commit only.
This PR adds the following to Delta Kernel Java:
JsonMetadataDomain.java: Introduces the base abstract class JsonMetadataDomain for metadata domains that use JSON as their configuration string. Concrete implementations, such as RowTrackingMetadataDomain, should extend this class to define their specific metadata domain behavior. This class provides
capabilities for serialize to/deserialize from the configuration string
construct a DomainMetadata action for committing
extract a specific metadata domain instance from a snapshot
RowTrackingMetadataDomain.java: Implements the metadata domain delta.rowTracking. It manages row ID high water mark, which will be used in the future for assigning fresh row IDs.
How was this patch tested?
Added tests covering the functionalities of JsonMetadataDomain and RowTrackingMetadataDomain in DomainMetadataSuite.scala.
Which Delta project/connector is this regarding?
Description
This PR builds on the base changes for introducing DomainMetadata into the Kernel, which are not yet merged. For changes specific to this PR, please refer to the last commit only.
This PR adds the following to Delta Kernel Java:
JsonMetadataDomain.java: Introduces the base abstract classJsonMetadataDomainfor metadata domains that use JSON as their configuration string. Concrete implementations, such asRowTrackingMetadataDomain, should extend this class to define their specific metadata domain behavior. This class providesDomainMetadataaction for committingRowTrackingMetadataDomain.java: Implements the metadata domaindelta.rowTracking. It manages row ID high water mark, which will be used in the future for assigning fresh row IDs.How was this patch tested?
Added tests covering the functionalities of
JsonMetadataDomainandRowTrackingMetadataDomaininDomainMetadataSuite.scala.Does this PR introduce any user-facing changes?
No.