search

rapidsai / distributed-join

Other
19 stars 12 forks source link

nvcomp integration #43

Closed gaohao95 closed 3 years ago

gaohao95 commented 3 years ago

This PR add an argument compression to distributed_inner_join, such that if it is set to true, the input buffers will be compressed before sending to remote GPUs, and decompressed after receiving. The compression and decompression are performed by nvcomp. Only cascaded compression on integer columns will be supported.

gaohao95 commented 3 years ago

Also, have you tested this code? How?

Thanks for the review! The modified tests have not been pushed yet. My plan is to extend the test cases for both the uncompressed version and the compressed version.