Closed kingwind94 closed 4 years ago
Ok, I think we do not need the select pushdown for now, only the project pushdown as you describe.
Do we need to verify the cols are same length here before pushback, will it throw an error?
Just did the check, though can't test this function for now still.
I think we also need to check there are no selection predicates.
I don't think we should do the selection predicate check. Because it will pick up those columns in selection predicates and push them back to the vector for the output table.
This looks good, thanks for the fixes and some refactoring as well. Can we remove the commented out blocks? Then let's merge.
just did it
Adds option for vertical partitioning (columnar) to pushdown a projection of all cols referenced by a query, and removes the select pushdown, instead applies selects on the client side after full tuple reconstruction from the projected cols. This PR implements a few example list ops for now, which may change when we move to Arrow compute API. Also optimizes result table creation by projecting full cols for project queries when no selects are present, instead of using the Arrow chunked array builder. Fixes a bug with chunk indexes.