Closed onesuper closed 8 years ago
Apache Sqoop uses a trick to maximize the concurrency.
https://abgaik.wordpress.com/2013/06/09/splitting-data-in-apache-sqoop/
It relies on the distribution of numeric values.
We could use this trick to fully utilize the network bandwidth when uploading records to ODPS.
cc @lyman
Done. The code is under /example
Apache Sqoop uses a trick to maximize the concurrency.
https://abgaik.wordpress.com/2013/06/09/splitting-data-in-apache-sqoop/
It relies on the distribution of numeric values.
We could use this trick to fully utilize the network bandwidth when uploading records to ODPS.
cc @lyman