ClickHouse / spark-clickhouse-connector

Spark ClickHouse Connector build on DataSourceV2 API
https://clickhouse.com/docs/en/integrations/apache-spark
Apache License 2.0
188 stars 66 forks source link

Spark: Cached ClickHouse table is not used by SQL #274

Closed zhouyifan279 closed 1 year ago

zhouyifan279 commented 1 year ago

Run Spark SQL:

cache table clickhosue.db.tbl;
explain select * from clickhosue.db.tbl;
== Physical Plan ==
*(1) Project [i#776]
+- BatchScan[i#776] class xenon.clickhouse.read.ClickHouseBatchScan RuntimeFilters: []

From the Physical Plan, we found the cached table was not used by this query.