[Feature] Write your feature request here!🚀

[kira-lin]

Here we will maintain a list of requested features, to make it easier to track.

• [ ] 1. Add custom resource requirements for RayDP executor actors. This has been implemented for RayDP master actors thanks to @pang-wu . Related issue #119
• [ ] 2. Release conda package. Related issue #124
• [ ] 3. Support Ray Dataset Pipelining with Spark Dataframe
• [ ] 4. Support streaming. Spark Streaming/Flink/Kafka? Need more use cases. Related issue #213 #291
• [ ] 5. Provide a guide for using RayDP in Ray on Kubernetes. Related issue #305
• [ ] 6. Integration with Spark native SQL engine Gluten

Feel free to write yours below! Also welcome to share your use case or scenario of RayDP, this will help us better understand your feature request, and how to further optimize performance, thanks!

[pang-wu]

@kira-lin Is it possible to make the point you listed with numbers so we can up vote? I would vote for No. 1, 3, 4

[kira-lin]

@kira-lin Is it possible to make the point you listed with numbers so we can up vote? I would vote for No. 1, 3, 4

Thanks, good idea!

[kira-lin]

@pang-wu Regarding to issue No. 4, are you currently dealing with streaming data in your workflow? What tool should RayDP provide in your expectation?