ClickHouse / clickhouse-jdbc-bridge

A JDBC proxy from ClickHouse to external databases
Apache License 2.0
167 stars 60 forks source link

due to: Not able to find suitable driver for datasource #142

Open nickscut opened 2 years ago

nickscut commented 2 years ago

image

select from jdbc('jdbc:mysql://rm-xxx.mysql.rds.aliyuncs.com:3306/davinci?useUnicode=true&user=xx&password=xxx','xxx','select from cron_job');

Such SQL will report an error

select from jdbc('mysql5', 'xxx', 'select from cron_job')

Such SQL runs fine

Our mysql is dynamic, there is no way to configure it in datasources in advance

nickscut commented 2 years ago

select from jdbc('jdbc:mysql://rm-xxx.mysql.rds.aliyuncs.com:3306/xxx?useUnicode=true&user=xx&password=xxx','xxx','select from cron_job');

Why does this SQL run fail ?

nickscut commented 2 years ago

image I follow this documentation using

zhicwu commented 2 years ago

select from jdbc('jdbc:mysql://rm-xxx.mysql.rds.aliyuncs.com:3306/xxx?useUnicode=true&user=xx&password=xxx','xxx','select from cron_job');

Why does this SQL run fail ?

Have you tried user:password@host_or_host_sublist as mentioned at here?

Update: and make sure user name and password are URL encoded.

nickscut commented 2 years ago

select from jdbc('mysql://davinci:Davinci123@rm-xx.mysql.rds.aliyuncs.com:3306/davinci','davinci','select from cron_job');

I make sure user name and password are URL encoded.

image

zhicwu commented 2 years ago

Did you forgot jdbc: prefix? It should be jdbc:mysql:... not mysql:...?

nickscut commented 2 years ago

select from jdbc('jdbc:mysql://davinci:Davinci123@rm-xxx.mysql.rds.aliyuncs.com:3306/davinci?useUnicode=true','davinci','select from cron_job');

image

zhicwu commented 2 years ago

The URL seems correct. Where did you put the MySQL JDBC driver?

nickscut commented 2 years ago

image Do I have to define such a json file?

nickscut commented 2 years ago

We have a lot of mysql, and they are added dynamically, there is no way to define the json file in advance

nickscut commented 2 years ago

If the json file is not defined, how to put the driver in the bridge?

nickscut commented 2 years ago

select from jdbc('mysql5', 'davinci', 'select from cron_job')

This SQL statement works fine,mysql5 is configured in advance in the mysql5.json file

but:We have a lot of mysql, and they are added dynamically, there is no way to define the json file in advance

zhicwu commented 2 years ago

Do I have to define such a json file?

No, you don't. But named datasource in general provides better security and performance. Also, you don't have to restart JDBC bridge each time adding a new JSON file - it will reload automatically.

If the json file is not defined, how to put the driver in the bridge?

You can put the driver in lib or any other directory and make sure it's included in CLASSPATH. Or, you can simply use docker-compose to start the environment and make sure you put JDBC drivers under $DRIVERS_DIR.

  jdbc-bridge:
    image: clickhouse/jdbc-bridge:2.0
    hostname: jdbc-bridge
    # In general you don't need to define any environment variable
    # Below are all default settings just for demonstration
    environment:
      CONFIG_DIR: config # configuration directory
      SERIAL_MODE: "false" # whether run query in serial mode or not
      HTTPD_CONFIG_FILE: httpd.json # httpd configuration file
      SERVER_CONFIG_FILE: server.json # server configuration file
      VERTX_CONFIG_FILE: vertx.json # vertx configuration file
      DATASOURCE_CONFIG_DIR: datasources # named datasource directory
      DRIVER_DIR: drivers # driver directory
      EXTENSION_DIR: extensions # extension directory
      QUERY_CONFIG_DIR: queries # named query directory
      CUSTOM_DRIVER_LOADER: "true" # whether use custom driver loader or not
      JDBC_BRIDGE_JVM_OPTS: # use CPU and memory allocated by container

    # You may want to keep datasources, queries, SQL scripts, and maybe drivers in a git repo
    volumes:
      - ./jdbc-bridge/config:/app/config
      - ./jdbc-bridge/drivers:/app/drivers
      - ./jdbc-bridge/scripts:/app/scripts
    mem_limit: 512m
    restart: always
nickscut commented 2 years ago

Does it support K8S deployment?

Is there a complete chart file?

nickscut commented 2 years ago

@zhicwu Does bridge support k8s deployment? Is there a complete chart file?

nickscut commented 2 years ago

image

It seems that there is no official k8s chart file for the time being

zhicwu commented 2 years ago

Yes, but it should be able to come up with one quickly based on docker-compose file - please feel free to submit a pull request.