ClickHouse / spark-clickhouse-connector

Spark ClickHouse Connector build on DataSourceV2 API
https://clickhouse.com/docs/en/integrations/apache-spark
Apache License 2.0
176 stars 63 forks source link

no ssl support #304

Closed paf91 closed 3 months ago

paf91 commented 3 months ago

Master branch and other releases has no ssl support. When I use these options:

spark = SparkSession.builder\
.config("spark.sql.catalog.clickhouse", "xenon.clickhouse.ClickHouseCatalog")\
.config("spark.sql.catalog.clickhouse.host", "test-server-01")\
.config("spark.sql.catalog.clickhouse.protocol", "https")\
.config("spark.sql.catalog.clickhouse.http_port", "8443")\
.config("spark.sql.catalog.clickhouse.user", "test-user")\
.config("spark.sql.catalog.clickhouse.password", "passwd")\
.config("spark.sql.catalog.clickhouse.option.ssl", "true")\
.getOrCreate()

Spark gives me this error:

Caused by: com.clickhouse.client.ClickHouseException: The target server failed to respond, server ClickHouseNode [uri=test-server-1:8443/default]@1830146098
    at com.clickhouse.client.ClickHouseException.of(ClickHouseException.java:169)
    at com.clickhouse.client.AbstractClient.lambda$execute$0(AbstractClient.java:275)
    at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    ... 1 more

Looking at clickhouse logs i see this:

2024.04.06 22:08:28.019551 [ 1389940 ] {} <Error> ServerErrorHandler: Code: 210. DB::NetException: SSL Exception: error:1000009c:SSL routines:OPENSSL_internal:HTTP_REQUEST, while reading from socket (some_spark_executor_node:15855). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):
0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c6d5d7b in /usr/bin/clickhouse
1. DB::NetException::NetException<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x000000000c975627 in /usr/bin/clickhouse
2. DB::ReadBufferFromPocoSocket::nextImpl() @ 0x000000000c975249 in /usr/bin/clickhouse
3. DB::HTTPServerRequest::HTTPServerRequest(std::shared_ptr<DB::IHTTPContext>, DB::HTTPServerResponse&, Poco::Net::HTTPServerSession&) @ 0x0000000012697fce in /usr/bin/clickhouse
4. DB::HTTPServerConnection::run() @ 0x00000000126965a4 in /usr/bin/clickhouse
5. Poco::Net::TCPServerConnection::start() @ 0x00000000150f4e52 in /usr/bin/clickhouse
6. Poco::Net::TCPServerDispatcher::run() @ 0x00000000150f5c51 in /usr/bin/clickhouse
7. Poco::PooledThread::run() @ 0x00000000151ece67 in /usr/bin/clickhouse
8. Poco::ThreadImpl::runnableEntry(void*) @ 0x00000000151eb45c in /usr/bin/clickhouse
9. ? @ 0x00007f2c94dbcac3 in ?
10. ? @ 0x00007f2c94e4e850 in ?
 (version 23.12.2.59 (official build))

Even though we have ssl in docs examples: https://github.com/housepower/spark-clickhouse-connector/blob/71d8e01a31ce83f12adba98ceb57daa01429ffcd/docs/configurations/01_catalog_configurations.md?plain=1#L67

paf91 commented 3 months ago

Was misconfiguration