Closed nmandery closed 2 years ago
Split RecordBatches into small units suitable for streaming to stay bellow the GRPC messages size limit. This could be accomplished by slicing arrow Arrays and assembling the ArrayRefs into new recordbatches. This should also be zero-copy.
https://github.com/nmandery/route3/blob/06f2aa545430764c3b92e80fbe83f8afce33ca25/route3_road/src/server/util.rs#L70
Split RecordBatches into small units suitable for streaming to stay bellow the GRPC messages size limit. This could be accomplished by slicing arrow Arrays and assembling the ArrayRefs into new recordbatches. This should also be zero-copy.
https://github.com/nmandery/route3/blob/06f2aa545430764c3b92e80fbe83f8afce33ca25/route3_road/src/server/util.rs#L70