Eventual-Inc / Daft

Distributed DataFrame for Python designed for the cloud, powered by Rust
https://getdaft.io
Apache License 2.0
2.17k stars 145 forks source link

Clarify URI syntax for connecting to ray cluster #2927

Open kevinjqliu opened 1 week ago

kevinjqliu commented 1 week ago

Describe the bug Ray exposes two ports, 6379 and 10001, depending on which port is used, the syntax for daft.context.set_runner_ray is different.

For example: Both of these works

daft.context.set_runner_ray(address="127.0.0.1:6379")
daft.context.set_runner_ray(address="ray://127.0.0.1:10001")

But this does not

daft.context.set_runner_ray(address="ray://127.0.0.1:6379")

This might be a Ray thing, but I think it'll make sense to clarify it in the Daft docs. Esp, since there are lots of examples using the ray:// prefix https://github.com/search?q=repo%3AEventual-Inc%2FDaft+%22ray%3A%2F%2F%22&type=code

To Reproduce Code above

Expected behavior As a user, I would want Ray to work regardless of which "style" I specify.

daft.context.set_runner_ray(address="127.0.0.1:6379")
daft.context.set_runner_ray(address="127.0.0.1:10001")
daft.context.set_runner_ray(address="ray://127.0.0.1:6379")
daft.context.set_runner_ray(address="ray://127.0.0.1:10001")

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

Smartphone (please complete the following information):

Additional context Add any other context about the problem here.

samster25 commented 1 week ago

assigning @raunakab!