[X] I had searched in the issues and found no similar feature requirement.
Description
Cross languge tasks/actors can only return data as results of functions. In some cases, this is not enough. It would be more convenient if Java can return object ref to python, because the worker that returns data does not have to have the data locally. Of course, such object cannot be an arbitrary java object. In our application, data is byte array, so there should be no problem with serialization. Currently, java cannot return object ref(s) to python. Object ID can be passed, since it is byte array, but ref = ray.ObjectRef(objectId); ray.get(ref) hangs.
Use case
For example, in raydp, data are put into ray object store by java executors, but a designated actor, called holder, owns all the data since executors may exit at any time. Python application wants to get data from the holder, but holder cannot return the data directly because data is actually distributed on many nodes. A better solution is to pass an object ref to python, and any python worker can get the object.
kira-lin
commented
3 years ago
Search before asking
Description
Cross languge tasks/actors can only return data as results of functions. In some cases, this is not enough. It would be more convenient if Java can return object ref to python, because the worker that returns data does not have to have the data locally. Of course, such object cannot be an arbitrary java object. In our application, data is byte array, so there should be no problem with serialization. Currently, java cannot return object ref(s) to python. Object ID can be passed, since it is byte array, but
ref = ray.ObjectRef(objectId); ray.get(ref)hangs.Use case
For example, in raydp, data are put into ray object store by java executors, but a designated actor, called holder, owns all the data since executors may exit at any time. Python application wants to get data from the holder, but holder cannot return the data directly because data is actually distributed on many nodes. A better solution is to pass an object ref to python, and any python worker can get the object.
Related issues
No response
Are you willing to submit a PR?