Ray state handler - Githubissues

ray-project / ray_beam_runner

Ray-based Apache Beam runner

Apache License 2.0

41 stars 12 forks source link

Ray state handler #6

Closed pabloem closed 2 years ago

pabloem commented 2 years ago

This implements the sdk_worker.StateHandler interface with Ray to provide a simple, single-actor holder for Beam streaming state. Objects of this StateHandler can then be passed to instances of worker handlers which will run workloads as Ray tasks.

Open questions

Maybe I should be adding more docstrings to the classes?
Maybe we should remove the license header? What license header do we want?

fyi @pdames @jjyao

pabloem commented 2 years ago

i guess we don't have to start this way. I'm just trying to start prototyping : )

pabloem commented 2 years ago

the question also remains open on how we convert PCollections into Datasets.... and also what this means for streaming. Maybe we can chat about this on Wednesday

iasoon commented 2 years ago

Can this PR be closed?

pabloem commented 2 years ago

yes thanks!