search

ClickHouse / spark-clickhouse-connector

Spark ClickHouse Connector build on DataSourceV2 API
https://clickhouse.com/docs/en/integrations/apache-spark
Apache License 2.0
188 stars 66 forks source link

Spark: Support `tcp_port` in catalog option #223

Closed camper42 closed 1 year ago

pan3793 commented 1 year ago

Thanks for fixing this, does the upstream ClickHouse Java Client support TCP protocol now?

camper42 commented 1 year ago

clickhouse-cli-client(wrapper of ClickHouse native command-line client) was added in 0.3.2-patch10, clickhouse-tcp-client will be available in 0.5

from 0.5.0, we use 0.3.2-patch11, it is supported if "added in 0.3.2-patch10" means "added from 0.3.2-patch10"

camper42 commented 1 year ago

clickhouse-cli-client(wrapper of ClickHouse native command-line client) was added in 0.3.2-patch10, clickhouse-tcp-client will be available in 0.5

if use tcp, we need a clickhouse cli in our environment before clickhouse-java 0.5

camper42 commented 1 year ago

we have performance issue when use http protocol.

we set batchSize to 1000 to avoid xenon.clickhouse.exception.CHServerException: [HTTP]user@host:8123}/default [1002] Error writing request body to server, but it's 2x slower vs 0.4.0 connect+grpc.

we want to give tcp a try