Kyligence / ClickHouse

ClickHouse® is a free analytics DBMS for big data
https://clickhouse.com
Apache License 2.0
14 stars 17 forks source link

[GLUTEN-1914] Allow nullable complex type when convert arrow column to ch column. #450

Closed taiyang-li closed 1 year ago

taiyang-li commented 1 year ago

Changelog category (leave one):

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Close https://github.com/oap-project/gluten/issues/1914

原来的实现:当将arrow column转化为ch column时,如果arrow type是负责类型(list/map/struct), 则不抽取arrow column的null map, 因为ch中复杂类型都是非nullable的。

现在的实现:为了保持与gluten兼容,即允许复杂类型为nullable, 修改代码,使得arrow type为复杂类型时,依然抽取null map和data column, 构成对应ch的nullable column

kyligence-git commented 1 year ago

Can one of the admins verify this patch?

taiyang-li commented 1 year ago

@baibaichen @liuneng1994 @binmahone can you review this pr