[Umbrella] [Feature] Supported More CDC Ingestion

[MonsterChenzhuo]

Search before asking

• [X] I searched in the issues and found nothing similar.

Motivation

Currently, Paimon supports CDC Ingestion for MySQL, Kafka, and MongoDB. I'd like to further implement data ingestion for more databases using CDC. links:https://paimon.apache.org/docs/master/how-to/cdc-ingestion.

• [ ] Supported OceanBase cdc
• [ ] Supported Postgres cdc @s7monk
• [ ] Supported SOLServer cdc

Solution

No response

Anything else?

No response

Are you willing to submit a PR?

• [ ] I'm willing to submit a PR!

[SteNicholas]

@MonsterChenzhuo, why does Paimon support more source? IMO, the Paimon is mained to support streaming lakehouse, not connector of different source .

[MonsterChenzhuo]

@SteNicholas I apologize for not being clear before. This question is about implementing write support for data lakes in a similar way to MySQL's Action. links:https://paimon.apache.org/docs/master/how-to/cdc-ingestion.

Currently, the databases I've listed are relatively stable in terms of Flink CDC synchronization support. Additionally, my company has this specific requirement. Therefore, I'd like to implement it and contribute to the community.

[Alibaba-HZY]

oracle cdc is also needed， because oracle to ogg-json is paid and expensive

[MonsterChenzhuo]

@Alibaba-HZY ok

[s7monk]

I can complete Postgres cdc. At present, I only need to run through the test. The local test cluster can run through. WIP

[MonsterChenzhuo]

@s7monk ok

[fuyun2024]

We have used Flink Oracle CDC before, but there are still many issues with the lower version of Debezium. I do not recommend supporting it at this time.

[hwhscut]

Is support for Oracle and SQL Server under development?

[medivh511]

Any plan for this? I hope we could do the oracle cdc directly from paimon