more responsive, unordered, run_to_results

[GoogleCodeExporter]

Currently, Executor.run_to_results iterates through the FutureList, waiting
for each future to finish *before* continuing with the next one. The
results are returned in the same order as the list of input tasks.

Please could a generator-method be made that yields when *any* future
finishes (and possibly return results in a different order from the
inputs). This is quite a common pattern; however at the moment I'm having
to implement this using the hack below (since FutureList has no way to keep
track of what has already been yielded), which doesn't scale.

-            for future in fs:
-                if timeout is None:
-                    yield future.result()
-                else:
-                    yield future.result(end_time - time.time())
+            yielded = [False] * len(fs)
+            while False in yielded:
+                fs.wait(None if timeout is None else end_time -
time.time(), return_when=FIRST_COMPLETED)
+                for i, future in enumerate(fs):
+                    future._condition.acquire()
+                    if future._state == FINISHED and not yielded[i]:
+                        yielded[i] = True
+                        print "yielded #%s (%s total)" % (i, len(fs))
+                        yield future.result()
+                    future._condition.release()
+

A better method would be to have worker push FINISHED futures onto a
result_queue or something.

Original issue reported on code.google.com by infinity0x@gmail.com on 3 Apr 2010 at 8:04

[GoogleCodeExporter]

better implementation which is scalable but still quite messy:

    def run_to_results_any(self, calls):

        res_queue = Queue()

        def run(call):
            res = call()
            res_queue.put(res)
            return res

        fs = self.run_to_futures((partial(run, call) for call in calls),
return_when=RETURN_IMMEDIATELY)

        yielded = 0
        while yielded < len(fs):
            yield res_queue.get()
            yielded += 1

Original comment by infinity0x@gmail.com on 4 Apr 2010 at 2:36

[GoogleCodeExporter]

This code has been completely rewritten.

Original comment by Dublin...@gmail.com on 21 May 2010 at 1:19

[GoogleCodeExporter]

Original comment by brian.qu...@gmail.com on 14 Nov 2010 at 3:49

• Changed state: Done