search

substrait-io / substrait-java

Apache License 2.0
77 stars 72 forks source link

feat(spark): bitwise functions #309

Closed andrew-coleman closed 1 month ago

andrew-coleman commented 1 month ago

Adds support in the spark module for 8-bit and 16-bit integer types and for some bitwise functions. The catalyst optimizer generates expressions using these for certain query types.

Note that shift_right (and other bit shifting functions) might want to be considered for the core substrait function catalog, but it has been added here (temporarily?) as spark extension pending a longer term discussion/decision on their wider utility.

andrew-coleman commented 1 month ago

I'll fixup the conflict :)