Further test of map suggests it is still greedy. If you submit a large iterator, it gets consumed almost immediately, causing memory issues.
Here's a simple test. If map wasn't greedy, it would print "time to first result" almost immediately. But it prints when the processing is almost done.
import concurrent.futures
import time
import sys
def job(i):
return i
with concurrent.futures.ThreadPoolExecutor(max_workers=4) as executor:
s = 0
start_time = time.time()
for i in executor.map(job, xrange(1, 100000)):
if s == 0:
print "time to first result=%.3f s" % (time.time() - start_time)
s += i
print "time to finish=%.3f s" % (time.time() - start_time)
Further test of map suggests it is still greedy. If you submit a large iterator, it gets consumed almost immediately, causing memory issues.
Here's a simple test. If map wasn't greedy, it would print "time to first result" almost immediately. But it prints when the processing is almost done.