search

zhouqingqing / qpmodel

A Relational Optimizer and Executor
MIT License
66 stars 18 forks source link

Remove unnecessary redistribution node #169

Closed arzuschen closed 4 years ago

arzuschen commented 4 years ago

Previously join force redistribution on top of table scan for distributed table, but it is not necessary when the key match the distribution. Now the match is checked, only when the data distribution does not agree with the join keys, a redistribution node is added.

Also, data insertion is modified. Previously, even for distributed table all the rows are added onto the first partition (0), but with distributed table scan all have redistribution node on top, it is still working. Now redistribution may not be present, the original insertion must follow the distributed column.

Additional unittest is added to verify the change.

arzuschen commented 4 years ago

updated to take in consideration for different query dop. Since the dop for data insertion and dop for query may not be the same, need to force redistribution if the partition number is not the same.