To support the json format of aliyun Data Integration, DI in short.
The data in kafka collected by DI from mysql or oracle, are not in standard format. Not canal-json or debezium json. I would like call it aliyun-json.
We want to sink the data to paimon from kafka directly, with the kafka_sync_database/table action. So the aliyun-json must be supported, and parsed into cdc record.
Tests
Supplied with the commit.
API and Format
No.
Documentation
When used to process the data collected by ID, with paimon-flink-action, it just need to specify the json format as the following:
[flink]
Purpose
Linked issue: close #4529
To support the json format of aliyun Data Integration, DI in short. The data in kafka collected by DI from mysql or oracle, are not in standard format. Not canal-json or debezium json. I would like call it aliyun-json.
We want to sink the data to paimon from kafka directly, with the kafka_sync_database/table action. So the aliyun-json must be supported, and parsed into cdc record.
Tests
Supplied with the commit.
API and Format
No.
Documentation
When used to process the data collected by ID, with paimon-flink-action, it just need to specify the json format as the following: