Netflix / metacat

Apache License 2.0
1.61k stars 281 forks source link

add ParentChildRelationSqlService #595

Closed stevie9868 closed 4 months ago

stevie9868 commented 5 months ago

PR Summary: Introduction of Parent-Child Relationship Metadata in Metacat

This pull request introduces metadata to manage parent-child relationships within Metacat. The key invariants for these relationships are:

The implementation is integrated into the open-source Metacat project, as parent-child relationships will be a fundamental concept, especially for tables like Iceberg.

Important Classes:

  1. MySQLParentChildRelMetadataService.java:

    • Provides CRUD services to manipulate the parent-child relationship metadata.
  2. TableServiceImpl:

    • Integrates MySQLParentChildRelMetadataService into the table management workflow.

    Operations:

    • Create:

      • The parent-child relationship is always established before the table is created. If table creation fails, the parent-child relationship is then deleted.
      • This order has two advantages:
        1. Reverting changes to the parent-child relationship is less destructive.
        2. Since both the table name and UUID are stored, it is safe to delete the record if necessary.
    • Rename:

      • Similar to the create operation, the parent-child relationship is renamed before the table is renamed. And if table renamed fail, we will try to rename the parent-child relation back.
    • Drop:

      • For the drop operation, the table is always dropped first, followed by the parent-child relationship if applicable.
      • If the parent-child relationship cannot be dropped, it will act as a dangling reference.
    • Get:

      • We will always fetch the parent child relation from MySQLParentChildRelMetadataService and then insert it into the definitionMetadata. For now we will only populate the field for child table. For parent to get the list of children, we expose a search endpoint.
  3. metacat.sql:

    • Define the schema of the parent child relation schema