search

apache / dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
https://dolphinscheduler.apache.org/
Apache License 2.0
12.75k stars 4.59k forks source link

[Improvement][DataSource] Support multi-host jdbc url #15119

Closed NJCrazyRen closed 9 months ago

NJCrazyRen commented 11 months ago

Search before asking

Description

Some distributed databases such as MPP or OLAP may have multi-host jdbc connection urls like "jdbc: xxx://ip1:port1,ip2:port2,ip3:port3/db?params". Therefore, it is recommended to improve the data source configuration to support such multiple hosts jdbc url.

Are you willing to submit a PR?

Code of Conduct

xinxingi commented 11 months ago

The multi-host URL configuration of different data sources may be different. Which types of data sources do you expect to implement this feature? Currently, the doris data source supports the use of multiple IP methods to link to the database to achieve functions similar to automatic load balancing.

NJCrazyRen commented 11 months ago

We are using GaussDB(DWS) as datawarehouse on Huawei Cloud, which has a port number after each host in its URL, even if the port number is actually the same. For example, "jdbc:gaussdb://10.x.x.1:8000,10.x.x.2:8000,10.x.x.3:8000/db"

xinxingi commented 11 months ago

@NJCrazyRen Since I don’t have Huawei Cloud’s data warehouse for verification, please help me verify it. jdbc: gaussdb:loadbalance://[host1][:port],[host2][:port][,[host3][:port]]...[/[database]] Is it possible to connect to the database using jdbc?

https://dev.mysql.com/doc/connector-j/en/connector-j-usagenotes-j2ee-concepts-managing-load-balanced-connections.html

xinxingi commented 11 months ago

You can use a temporary solution first and include a layer of nginx to temporarily accomplish what you want.

NJCrazyRen commented 11 months ago

Since I don’t have Huawei Cloud’s data warehouse for verification, please help me verify it. jdbc: gaussdb:loadbalance://[host1][:port],[host2][:port][,[host3][:port]]...[/[database]] Is it possible to connect to the database using jdbc?

Unfortunately there's no text about this url format in GaussDB offical documents.

github-actions[bot] commented 10 months ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] commented 9 months ago

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.