open-metadata / OpenMetadata

OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.
https://open-metadata.org
Apache License 2.0
4.77k stars 920 forks source link

Reorganize OpenMetadata Python SDK #16428

Open TeddyCr opened 1 month ago

TeddyCr commented 1 month ago

Context Python SDK for OpenMetadata contains many different operations for interacting with the OpenMetadata API.

Generally speaking, operations are separated based on resources, and the entire Ometa class uses inheritance.

While this proves convenient, it also creates noise for user as they can see a mix of operations when instantiating the Ometa object.

Suggested Approach To streamline this approach we suggest different improvements:

pmbrull commented 1 month ago

brain dump

# "PRIVATE"
class _OmetaTestClient:

    def __init__(config, Core)
          config = config

    def upload_test()
         ...

    def ...
       create_or_update/get/list ->> This is from core

# "PUBLIC"
class OmetaCore:

     def __init__(config)
          config = config

     ## DOCS
     def get_...
     def create_or_update
     def list...
     ## !DOCS

class OmetaAPI:
     core = OmetaCore()
     _test = OmetaTestClient(core)

     def list = core.list() ->> define alias

api = OmetaAPI()

api.core.list?
api.list ->>

api._test.upload
api._lineage.get

# "Interal clients"
ometa.__test.upload_tests()

# "External clients"
TestClient()

class OmetaTestClient(OmetaCore)

tables = Table(client)
Listed = tables.list()

Ometa -> list, get put
TestClient -> upload test
TableClient -> upload Sample Data?