Open Mrart opened 1 month ago
Implementation steps:
1.[Done] Catalog store plugin .catalog store plugin 2.[Done] Catalog curd.Introduce catalog managed 3.[Doing] Streampark start job ship catalogstore plugin ship catalog plugin. 4.[To begin] Catalog managed frontend. 5.[Doing] Catalog database/table managed (backend/Frontend)
Looks good
@wolfboys All the backend code has been pushed, please review it ASAP.
Search before asking
Motivation
There are many internal business database systems, and the number and variety of upstream and downstream databases bring a lot of management difficulties. At the same time, most of the current business systems need to connect to Flink CDC for data synchronization. Flink itself supports metadata management, Streampark does not implement metadata management function at present, in order to improve the convenience of users to use data, we designed the metadata management function.
Design Detail
Product design
catalog management refers to the catalog management design of Alibaba Cloud real-time computing products, including jdbc catalog, hive catalog, paimon catalog, etc. jdbc catalog(mysql/pgsql)
hive catalog
paimong catalog
Catalog implementation detailed design
As shown
Design of catalog information table in streampark
Compatibility, Deprecation, and Migration Plan
No response
Test Plan
No response
Code of Conduct
References