search

oap-project / raydp

RayDP provides simple APIs for running Spark on Ray and integrating Spark with AI libraries.
Apache License 2.0
315 stars 69 forks source link

Slack Group for the commiters & maintainers #415

Open mehulbatra-d11 opened 1 month ago

mehulbatra-d11 commented 1 month ago

RayDP Community, Our dream11 team frequently utilizes RayDP to run and leverage Spark on top of Ray, and we're enthusiastic about contributing to its development. However, we've noticed some challenges in getting our pull requests reviewed and new features implemented. To address this, we'd like to propose establishing a more streamlined collaboration process. Some ideas include:

We believe these steps could significantly reduce friction in the development process and help advance the project more efficiently. We're open to discussing these suggestions and any other ideas the community might have. Let's work together to make RayDP even better! Looking forward to your thoughts

mehulbatra-d11 commented 1 month ago

@kira-lin @ConeyLiu @carsonwang

carsonwang commented 1 month ago

Hi @mehulbatraa, thanks for the suggestions! There is a Slack channel #spark-on-ray in Ray's slack. Sorry for slow review on the PRs because people were prioritized on something else recently. Can you please let us know the PRs and features contributed by your team and any future development plans?

mehulbatra-d11 commented 1 month ago

Hi @carsonwang and team thanks for the slack channel heads-up we will use that!

@raviranak and his team have been actively contributing to this project for an extended period. Their recent work includes: https://github.com/oap-project/raydp/pull/396 https://github.com/oap-project/raydp/pull/395 https://github.com/oap-project/raydp/pull/391 https://github.com/oap-project/raydp/pull/389

Issues Raised:

https://github.com/oap-project/raydp/issues/412 https://github.com/oap-project/raydp/pull/407 https://github.com/oap-project/raydp/issues/405

Regarding our future plans, we intend to integrate raydp into the system, which will involve various developments and feature enhancements. I will provide a short-term roadmap in the near future outlining our immediate objectives. I hope this information is helpful.

It would be beneficial if your team could develop a comprehensive roadmap for the raydp project's future. This roadmap should outline

  1. The overall direction you envision for the project
  2. Key features you're planning to implement or enhance

Sharing this information could significantly contribute to the project's growth and help align efforts across the community. Would you be able to create and share such a roadmap?

carsonwang commented 1 month ago

Thanks for all the details! Currently we don't have a roadmap that you were requesting. But we'll support if you have the features and enhancements required in your environment. So welcome any contributions.