search

ClickHouse / spark-clickhouse-connector

Spark ClickHouse Connector build on DataSourceV2 API
https://clickhouse.com/docs/en/integrations/apache-spark
Apache License 2.0
176 stars 63 forks source link

Array and Map should not be wrapped in Nullable #303

Closed heartdance closed 3 months ago

heartdance commented 3 months ago

ClickHouse not support Array or Map wrapped in Nullable. ex: Array(Nullable(String)) is supported, but Nullable(Array(String)) is not supported.

pan3793 commented 3 months ago

Do you have the opportunity to add an e2e test for your change? And you only need to change the code in spark-3.5, there is a script to backport your change to lower Spark versions' module

heartdance commented 3 months ago

Do you have the opportunity to add an e2e test for your change? And you only need to change the code in spark-3.5, there is a script to backport your change to lower Spark versions' module

ok, added