Add Semaphore, Condition, Event

dask / distributed

A distributed task scheduler for Dask

https://distributed.dask.org

BSD 3-Clause "New" or "Revised" License

1.57k stars 718 forks source link

Add Semaphore, Condition, Event #2007

Open mrocklin opened 6 years ago

mrocklin commented 6 years ago

Dask currently has some distributed coordination primitives like Locks and Queues that match the API of normal local Locks and Queues. There are other primitives like these that we do not yet support, notable Semaphore, Condition, and Event.

These might be good training for people who want to become familiar with extending the dask.distributed system. The files lock.py, queues.py, and variable.py might be a good model for development here.

palash25 commented 6 years ago

Hi, I've been writing python for some time but I'm new to distributed computing. This seems like the right issue to get started. Can I work on this?

mrocklin commented 6 years ago

Yes! Anyone can work on any issue. You may want to familiarize yourself with http://distributed.readthedocs.io/en/latest/develop.html

On Tue, Jul 10, 2018 at 2:49 PM, Palash Nigam notifications@github.com wrote:

Hi, I've been writing python for some time but I'm new to distributed computing. This seems like the right issue to get started. Can I work on this?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/dask/distributed/issues/2007#issuecomment-403927924, or mute the thread https://github.com/notifications/unsubscribe-auth/AASszEyjQTLpoBFJ-s1yquemYgzN4Ipmks5uFPcpgaJpZM4UNOYM .

palash25 commented 6 years ago

Thanks will try to submit a PR soon

andersy005 commented 4 years ago

It appears that https://github.com/dask/distributed/pull/2690 is/was meant to implement one (Semaphore) of the missing features listed here. Are there any updates on the status of https://github.com/dask/distributed/pull/2690?
https://github.com/dask/distributed/pull/3397 mentions Events and Conditions. Is it the case that these two features have also been implemented already?

I am interested in working on the other features namely (Event, Condition), and wanted to make sure that I am not duplicating work that has already been done.

mrocklin commented 4 years ago

It appears that #2690 is/was meant to implement one (Semaphore) of the missing features listed here. Are there any updates on the status of #2690?

All communication is happening on github. I recommend checking in on that PR.

3397 mentions Events and Conditions. Is it the case that these two features have also been implemented already?

This is different. Dask uses Tornado/asyncio objects within its codebase. We would also like to expose distributed versions of these objects to users so that they can make their own applications.

andersy005 commented 4 years ago

All communication is happening on github. I recommend checking in on that PR.

👍 will do

This is different. Dask uses Tornado/asyncio objects within its codebase. We would also like to expose distributed versions of these objects to users so that they can make their own applications.

Thank you for the clarification!