[Core] Implement Ray based concurrent.futures.Executor

[Al12rs]

Description

https://docs.python.org/3/library/concurrent.futures.html#executor-objects

It would be very nice to have a ray based (so distributed) implementation of a concurrent.futures.Executor, an analogue of concurrent.futures.ProcessPoolExecutor simply working on a distributed cluster using Actors.

This would be similar to ray.util.multiprocessing.Pool with the important difference of supporting asyncio interfaces. One of the core strength of Ray is the asyncio support and this Feature request would continue Ray's progress on that front.

I found this issue #1883 where other users expressed the same request in the past, which was incorrectly closed in my opinion when cuncurrent.futures.Future API compliance was added, and the remaining part of the request was ignored.

Use case

The most useful use case would of course be for libraries and programs which already make use of the concurrent.futures.ProcessPoolExecutor interface for example. They would then be able to move to distributed by simply changing the import to use the ray version instead.

The other use case are the same as for ray.util.multiprocessing.Pool, where we would like to have Ray manage a pool of actors for us, but would like to be able to avoid blocking on results and instead use asyncio loop.run_in_executor() to be able to use await and handle many things concurrently.

What I personally would really like is the ability to have a managed pool of async actors which can all make use of asyncio to avoid blocking on ray.get calls and the like. This way the submitted remote tasks can await other results and in the meantime handle other tasks. This can currently only be implemented with actors but the responsibility of scaling the actors and scheduling the work to them is on the shoulders of the developers using Ray, instead of being offered by Ray itself.

[richardliaw]

Hey, that's a good point. It would certainly allow for a much faster port to distributed execution.

FYI, @pcmoritz @ericl @scv119 for API surface addition. cc @simon-mo for previous introduction of concurrent.futures.

[richardliaw]

BTW, assuming we are OK with the proposal, @Al12rs would you be open to pushing a PR?

[Al12rs]

I'm afraid I'm not confident to have the knowledge or the free time to contribute directly.

[asaacke]

Hello, I'm a senior CS undergraduate student and am looking to contribute to sharpen my skills and benefit the project. Could I be assigned to work on this task? Timeline would be working on this over the next month.

[richardliaw]

sure, go ahead and take it on! though keep in mind we may reassign this if there is no progress in 2 weeks :)

On Thu, Nov 10, 2022 at 7:49 AM asaacke @.***> wrote:

Hello, I'm a senior CS undergraduate student and am looking to contribute to sharpen my skills and benefit the project. Could I be assigned to work on this task? Timeline would be working on this over the next month.

— Reply to this email directly, view it on GitHub https://github.com/ray-project/ray/issues/29456#issuecomment-1310494467, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCRZZJ7A2IYBIDW6KKFX6LWHUKPPANCNFSM6AAAAAARJDYZRQ . You are receiving this because you commented.Message ID: @.***>

[jeicher]

Hi! Some of our team members at Tweag have been looking at this issue too. :) Hope that's alright.

[jeicher]

Hi @richardliaw 🙂, just pinging here to let you know that we have pushed a PR above ☝🏻

[judahrand]

44922 is currently open but seems to be waiting for review can anyone have a look (@jjyao maybe?)