Support Ray as executor

[c21]

Ray (https://github.com/ray-project/ray) becomes popular choice of running distributed Python ML applications. Its Python interface is easy to scale up the workload from local laptop to distributed cluster. It would be good to add Ray as an executor backend (and we are happy to contribute).

Some more info related in this topic:

• RAG embedding generation w/ Ray and Pinecone - https://www.anyscale.com/blog/rag-at-scale-10x-cheaper-embedding-computations-with-anyscale-and-pinecone
• Building RAG-based LLM Applications for Production w/ Ray - https://www.anyscale.com/blog/a-comprehensive-guide-for-building-rag-based-llm-applications-part-1

[guipenedo]

Thank you for your suggestion and specially for being willing to contribute! I thought ray would be more of an alternative (specially with ray data) rather than an environment where datatrove could run, but if you think you could add support as an executor (without making any changes to anything in the pipeline module), I'd be happy to have ray support in datatrove :)

[simplew2011]

A similar tool, data Juicer, developed ray-executor： https://github.com/alibaba/data-juicer/blob/main/data_juicer/core/ray_executor.py