search

dask / community

For general discussion and community planning. Discussion issues welcome.
20 stars 3 forks source link

Dask Enhancement Proposals #137

Open mrocklin opened 3 years ago

mrocklin commented 3 years ago

Hello everyone,

I would like for us to consider adopting an enhancement proposal process. Github issues and PRs are good for day-to-day feature development, but larger efforts tend to get a bit scattered among several issues/blogposts/in-person conversations.

Historically I tracked larger issues personally and served as the human reference/collection point for larger planning. As I've been less active over the last few months this role seems to have gone dormant. This has happened in other projects before. For example Numpy seemed to benefit from the NEP process when that started.

I'm inclined to start brain-dumping a bunch of larger thoughts in my head down (recommended by @crusaderky). These might serve as a first set of DEPs and a good use case to flesh out this process. I'm inclined to learn about and start with the NEP structure if no other ideas come up before that.

cc @shoyer @quasiben

quasiben commented 3 years ago

Thanks for proposing this. I agree that a process around how dask is enhanced would be good especially when compared with collections GH issues. Do you see this being limited to dask / distributed or generally across the dask ecosystem ?

jrbourbeau commented 3 years ago

Cross-referencing NEP-0 (see https://numpy.org/neps/nep-0000.html) which provides information on NumPy's NEP process

jakirkham commented 3 years ago

We did this at conda-forge as well. I think the tricky thing is this is a whole other aspect of work. The people raising the EPs care a lot about getting them in. However as these can be particularly meaty/hairy proposals, everyone else struggles with finding time to review the EPs, which can be hard if their employer doesn't care about the EP under consideration or even simply if there are competing demands for time. So it requires a concerted effort by all parties to keep these moving. Dedicating some time (particularly in high bandwidth settings) can help keep EPs moving. Though some EPs still fall by the wayside even after doing that. To be clear I'm not saying this is a bad idea (it's actually a good one), but there are some unique challenges with EPs that do need to be addressed to make sure the process is effective

mrocklin commented 3 years ago

I suspect that due to the density of full time employed folks on Dask that we will have different problems than what Conda-Forge experienced.

mmccarty commented 3 years ago

+1 for this! We have started using an enhancement proposal process heavily inspired by NEP internally at C1. Also, Prefect has a similar process PINs.