[clickhouse]seatunnel.connectors.clickhouse:how can i connect it with https

xpchen-fa commented 1 year ago

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened

由于clickhouse安装在安全集群上，测试用通过curl的方式访问https可以但是http不行，是否可以设置https连接方式

test

curl -k -- insecure 'https:/ip:port' Ok. curl -k -- insecure 'http:/ip:port' curl: (52) Empty reply from server

SeaTunnel Version

2.3.1

SeaTunnel Config

source {
  hive {
    sql = "select data_dt,row_key from mdm.temp1"
    table_name = "mdm.temp1" 
    metastore_uri=""
  }
}
transform {
  Sql{
    source_table_name = "mdm.temp1"
    query="select data_dt,row_key from mdm.temp1" 
    result_table_name = "fake"
  }
}
sink {
  Clickhouse{
    host = "ip:port"
    clickhouse.socket timeout = 100000
    database = "mdm" 
    table = "tmp"
    username = "name" 
    password = "pass"
  }
}

Running Command

bin/start-seatunnel-spark-2-connector-v2.sh --config config/batch.conf -m local -e client

Error Exception

ERROR|main Run SeaTunnel on spark failed. |org.apache.seatunne1.core.starter.spark.command.SparkTaskExecuteCommand.execute(SparkTaskExecuteCommand.java:62)
org.apache.seatunne1.connectors.seatunne1.clickhouse.exception.ClickhouseConnectorException:ErrorCode:[COMMON-09]， ErrorDescription:[Get table sch ema from upstream data failed] - Cannot get table schema from clickhouse
at org.apache.seatunnel.connectors.seatunne1.clickhouse.sink.client.ClickhouseProxy.getClickhouseTab1eSchema(ClickhouseProxy.java:144) 
at org.apache.seatunne1.connectors.seatunne1.clickhouse.sink.client.ClickhouseProxy.getclickhouseTab1eSchema(ClickhouseProxy.java:133) 
at org.apache.seatunne1.connectors.seatunne1.clickhouse.sink.client.clickhouseSink.prepare(ClickhouseSink.java:141)
at org.apache.seatunnel.core.starter.spark.execution.SinkExecuteProcessor.1ambda$initializePlugins$0(SinkExecuteProcessor.java:76) 
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) 
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) 
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) 
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) 
at java.uti1.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.seatunne1.core.starter.spark.execution.SinkExecuteProcessor.initializePlugins(SinkExecuteProcessor.java:87)
at org.apache.seatunne1.core.starter.spark.execution.SparkAbstractP1uginExecuteProcessor.<init>(5parkAbstractP1uginExecuteProcessor.java:49
at org.apache.seatunne1.core.starter.spark.execution.SinkExecuteProcessor.<init>(SinkExecuteProcessor.java:53) 
at org.apache.seatunne1.core.starter.spark.execution.SparkExecution.<init>(SparkExecution.java:66)
at org.apache.seatunne1.core.starter.spark.command.SparkTaskExecuteCommand.execute(SparkTaskExecuteCommand. java:59) at org.apache.seatunnel.core.starter.SeaTunne1.run(SeaTunnel.java:40)
at org.apache.seatunnel.core.starter.spark.SeaTunne1Spark.main(SeaTunne1Spark.java:35) 
at sun.reflect.NativeMethodAccessorImp1.invokeO(Native Method)
at sun.reflect.NativeMethodAccessorImp1.invoke(NativeMethodAccessorImp1.java:62)
at sun.reflect.DelegatingMethodAccessorImp1.invoke(DelegatingMethodAccessorImp1.java:43) 
at java.lang.reflect.Method.invokeMethod.java:498)
at org.apache.spark.dep1oy.SparkSubmit$.orq$apache$spark$dep1oy$SparkSubmit$$runMain(SparkSubmit.scala:762) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:190) 
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:215) 
at org.apache.spark.deplov.SparkSubmit$.main(SparkSubmit.scala:129) 
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: com.clickhouse.client.ClickHouseException:Unexpected end of file from server, server ClickHouseNode [uri=http://ip:port/mdm 
]@-1720695623
at com.clickhouse.client.ClickHouseException.of(ClickHouseException.java:168)
at com.clickhouse.client.Abstractclient.1ambda$execute$0(Abstractclient.java:291)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.iava:1590) 
at java.util.concurrent.ThreadPoo1Executor.runWorker(ThreadPoo1Executor.java:1149) 
at java.util.concurrent.ThreadPoo1Executor$Worker.run(ThreadPoolExecutor.java:624) 
at java.lang.Thread.run(Thread.iava:748)
Caused by: java.net.SocketException: Unexpected end of file from server
at sun.net.www.http.HttpClient.parseHTTPHeader(Httpclient.iava:851)

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] commented 1 year ago

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.

apache / seatunnel