Map is still greedy

[mathieulongtin]

Further test of map suggests it is still greedy. If you submit a large iterator, it gets consumed almost immediately, causing memory issues.

Here's a simple test. If map wasn't greedy, it would print "time to first result" almost immediately. But it prints when the processing is almost done.

import concurrent.futures
import time
import sys

def job(i):
    return i

with concurrent.futures.ThreadPoolExecutor(max_workers=4) as executor:
    s = 0
    start_time = time.time()
    for i in executor.map(job, xrange(1, 100000)):
        if s == 0:
            print "time to first result=%.3f s" % (time.time() - start_time)
        s += i
    print "time to finish=%.3f s" % (time.time() - start_time)

[agronholm]

Right, forgot to close this one as wontfix. Will revisit if upstream implements throttling.

[Axik]

What is a possible workaround for this?