apache / doris-kafka-connector

Kafka Connector for Apache Doris
https://doris.apache.org
Apache License 2.0
10 stars 9 forks source link

[Improve]Optimize to determine whether table in doris exists #26

Closed DongLiang-0 closed 3 months ago

DongLiang-0 commented 3 months ago

Since each partition under the topic has a sink task, when the corresponding connector starts, it will connect to doris to determine whether the table exists. When there are multiple topics, and each topic has multiple partitions, a large number of connections will be made to request doris at the moment of startup, causing the number of connections to exceed the limit.

This PR can reduce a repeated request to determine whether the database exists.