at15 / cs433

SJTU cs433 distributed programming
GNU General Public License v2.0
4 stars 2 forks source link

Output meta in sort reducer using multiple output #12

Closed at15 closed 8 years ago

at15 commented 8 years ago

currently, the meta for each partition is generated in the index building phase, however, if using mellowtech's bptree, it seems I need to traverse the data twice to get the min and max key, because iterator is needed for bulk loading, and bptree does not provide max and min key.

at15 commented 8 years ago

oops, no, it seems to be impossible to generate the range, unless there is some before after method

at15 commented 8 years ago

yep, the cleanup method is called when one reducer finished its job, so I can get the max and min key there.

at15 commented 8 years ago

finished in ddcd677c96b6f826273eed983f6d2b02fba3b274