Closed dockerzhang closed 1 year ago
@dockerzhang Please assign to me, thank you!
Sort module supports Apache Hudi. Apache Hudi is a popular streaming datalake platform. We should support Apache Hudi in sort module.
The design will follow following the document Sort Plugin and Manager Plugin
HudiLoadNode
, which inherits the LoadNode classHudiExtractNode
, which inherits the ExtractNode classCreating new file called Hudi
in inlong-sort/sort-connectors
, Adding new classes into the file:
HudiTableSink
HudiTableSource
HudiTableFactory
ConfigOptions
HudiCatalog
HudiCatalogFactory
(As Apache Hudi has already integrated Flink, this part will refer to the implementation of flink connector in Apache Hudi)
Follow the document Manager Plugin to extend extract node and load node
This is a good idea and the plan is all right, looking forward to your pr.
This issue is stale because it has been open for 60 days with no activity.
duplicated with #https://github.com/apache/inlong/issues/6781, close it.
Describe the proposal
Sort module supports Apache Hudi.
Task list
InLong Component
Other for not specified component
Are you willing to submit PR?
Code of Conduct