Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Julia is a high performance, high level language that is widely used for technical computing, machine learning, data science, etc. The feature request is to add minimal Language::JULIA support to the ray core, updating the protobuf messages, enums, and worker pool manager to allow the creation of workers to execute Julia tasks.
We have work in progress in a fork which adds this functionality, which we would like to upstream to the Ray trunk soon. We have some small changes to make there still but expect to wrap up that work in the next few weeks, at which point we can open a PR.
Note that we are not (yet) proposing to upstream our Julia language Ray runtime, Ray.jl since that is under much heavier development, not yet feature-complete, and with known major bugs/issues that we're actively working on. This is strictly limited to the changes necessary so that when someone launches a ray cluster, that cluster knows how to handle tasks with Language::JULIA, launching workers as needed and scheduling tasks appropriately.
Use case
We (Beacon Biosignals) use Julia for all our "quantitative computing" work, from ML/algorithm development and inference to analysis of large multi-channel timeseries data (think audio/video but brain signals), and would like to use Ray to deploy our workloads in the cloud. There currently is not a good solution to deploying and scaling distributed Julia workloads that does not require a large amount of fiddly and brittle manual work. Integrating with Ray core allows us to use all the nice work that's been done on the object manager, autoscaling, k8s deployment, etc. without having to implement and maintain those features entirely on our own.
From the ray side, supporting the Julia language means that other organizations/teams that are interested in using Julia (possibly in addition to other languages) can use Ray to manage their workloads, potentially increasing uptake of Ray.
Description
Julia is a high performance, high level language that is widely used for technical computing, machine learning, data science, etc. The feature request is to add minimal
Language::JULIA
support to the ray core, updating the protobuf messages, enums, and worker pool manager to allow the creation of workers to execute Julia tasks.We have work in progress in a fork which adds this functionality, which we would like to upstream to the Ray trunk soon. We have some small changes to make there still but expect to wrap up that work in the next few weeks, at which point we can open a PR.
Note that we are not (yet) proposing to upstream our Julia language Ray runtime, Ray.jl since that is under much heavier development, not yet feature-complete, and with known major bugs/issues that we're actively working on. This is strictly limited to the changes necessary so that when someone launches a ray cluster, that cluster knows how to handle tasks with
Language::JULIA
, launching workers as needed and scheduling tasks appropriately.Use case
We (Beacon Biosignals) use Julia for all our "quantitative computing" work, from ML/algorithm development and inference to analysis of large multi-channel timeseries data (think audio/video but brain signals), and would like to use Ray to deploy our workloads in the cloud. There currently is not a good solution to deploying and scaling distributed Julia workloads that does not require a large amount of fiddly and brittle manual work. Integrating with Ray core allows us to use all the nice work that's been done on the object manager, autoscaling, k8s deployment, etc. without having to implement and maintain those features entirely on our own.
From the ray side, supporting the Julia language means that other organizations/teams that are interested in using Julia (possibly in addition to other languages) can use Ray to manage their workloads, potentially increasing uptake of Ray.