currently we have a very basic implementation of the SortOperator and a very basic rule for cross-server sorting plan generator. It generates an sort exchange which connects a local sorting prior to exchange then follow by a remote sorting on a single machine
this is originally designed for large quantity sorting and streaming responses. but none of those were implemented in SortOperator
Improvement
for the current SortOperator implementation, the sort-exchange can be simplified to a normal exchange, and the local sort prior to exchange can be removed.
when data size is small, we should follow the 1st approach;
when data size is large, we should
(a) create a local sort before the exchange, (either via sort-exchange, or via additional sort operator);
(b) do a k-merge sort on the single-machine global sorting operator
(c) stream the results back (which is possible with k-merge)
currently we have a very basic implementation of the SortOperator and a very basic rule for cross-server sorting plan generator. It generates an sort exchange which connects a local sorting prior to exchange then follow by a remote sorting on a single machine this is originally designed for large quantity sorting and streaming responses. but none of those were implemented in SortOperator
Improvement