search

apache / iceberg-go

Apache Iceberg - Go
https://iceberg.apache.org/
Apache License 2.0
142 stars 34 forks source link

feat(table/scanner): Implement Arrow type promotion and conversion #174

Closed zeroshade closed 4 weeks ago

zeroshade commented 1 month ago

Builds on #168 which should get merged first.

Adds proper handling for ensuring small or large types for Arrow data along with implementing a function to convert/cast record batch columns to appropriate and uniform iceberg types based on the iceberg schema itself. Also implements an ExtractFieldIDs method for extracting the field ids that are necessary for a given filter expression.

zeroshade commented 1 month ago

CC @Fokko @nastra

zeroshade commented 1 month ago

@Fokko @nastra rebased and updated, ready for review now!