search

stharrold / ARCHIVED_bench_mapr

Compare performance of Disco with Hadoop as measured by HiBench Hadoop benchmark suite.
MIT License
0 stars 1 forks source link

data: make multithread #8

Open stharrold opened 10 years ago

stharrold commented 10 years ago

data download-extract-upload is ~0.1 GB/min. modularize download-extract-upload section to be multithreaded and managed.

stharrold commented 10 years ago

See "Programming Python" chapter on threads. Also see http://stackoverflow.com/questions/9874042/using-pythons-multiprocessing-module-to-execute-simultaneous-and-separate-seawa/9874484#9874484 and http://www.tutorialspoint.com/python/python_multithreading.htm