search

vmware-archive / quickstep

Quickstep Project
Apache License 2.0
27 stars 13 forks source link

Initial partition-aware changes for hash join. #217

Closed adalbertgerald closed 8 years ago

adalbertgerald commented 8 years ago

I have made some initial changes to the HashJoinOperator.

It seems that the HashJoin code has changed a lot from what we had for before, when Harshad and I worked on making it partition-aware for the quickstep group paper.

So, I would like to get some initial feedback from @hbdeshmukh and @zuyu before making any more changes.

NOTE: I haven't updated the tests yet and so the travis builds would fail.

zuyu commented 8 years ago

As of now, I have above comments. Thanks!

adalbertgerald commented 8 years ago

Thanks for your comments @zuyu and @harshad. I have updated the buildhash and the hashjoin operators. Can you please take a look?

adalbertgerald commented 8 years ago

Hi @hbdeshmukh, I have made the changes for partition-aware hash joins. Can you please take a look?

zuyu commented 8 years ago

@adalbertgerald I believe there is a fundamental design issue that we tie the partition info to close with NUMA support. So in future when we would extend the partition info to distributed workers (but no NUMA support), it needs significant rework.

Could you please come up with an approach that first check wether we have partition info, and then use different strategies to achieve partitioning? In this case, we do in the NUMA case. And in future, we may only use the distributed case. Thanks!

adalbertgerald commented 8 years ago

Reopened a new pull request #238.