xorbitsai / xoscar

Python actor framework for heterogeneous computing.
https://xoscar.dev
Apache License 2.0
89 stars 21 forks source link

FEAT: make it easy to define generator methods in Actor #82

Closed liunux4odoo closed 8 months ago

liunux4odoo commented 8 months ago

What do these changes do?

Generators can not be transferred directly between actors, users must wrap it to a iterator object referencing to the code of xinference ModelActor. But the wrapping codes are complex and need to rewrite them in every generators.

This PR provides a decorator: xo.generator, users can define generators as usual in a simple way:

import asyncio
import time
import xoscar as xo

address = "127.0.0.1:8000"

class WorkerActor(xo.StatelessActor):
    @xo.generator
    def chat(self):
        for x in "hello oscar by sync":
            yield x
            time.sleep(0.1)

    @xo.generator
    async def achat(self):
        for x in "hello oscar by async":
            yield x
            await asyncio.sleep(0.1)

    @classmethod
    def uid(cls):
        return "worker"

class SupervisorActor(xo.StatelessActor):
    @xo.generator
    async def chat(self):
        worker_actor: xo.ActorRef["WorkerActor"] = await xo.actor_ref(address=address, uid=WorkerActor.uid())
        yield "\nsync"
        async for x in await worker_actor.chat(): # this is much confused. I will suggest use async generators only.
            yield x

        yield "\nasync"
        async for x in await worker_actor.achat():
            yield x

    @classmethod
    def uid(cls):
        return "supervisor"

async def main():
    pool = await xo.create_actor_pool(address, 2)
    await xo.create_actor(WorkerActor, address=address, uid=WorkerActor.uid())
    superivsor_actor = await xo.create_actor(SupervisorActor, address=address, uid=SupervisorActor.uid())

    async for x in await superivsor_actor.chat():
        print(x)

    await pool.join()

asyncio.run(main())

Related issue number

Fixes #xxxx

Check code requirements

codecov[bot] commented 8 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (c7f3de4) 89.02% compared to head (c4d53a2) 88.62%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #82 +/- ## ========================================== - Coverage 89.02% 88.62% -0.41% ========================================== Files 48 47 -1 Lines 4010 3920 -90 Branches 764 751 -13 ========================================== - Hits 3570 3474 -96 - Misses 355 360 +5 - Partials 85 86 +1 ``` | [Flag](https://app.codecov.io/gh/xorbitsai/xoscar/pull/82/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xorbitsai) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/xorbitsai/xoscar/pull/82/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xorbitsai) | `88.54% <ø> (-0.31%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xorbitsai#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codingl2k1 commented 8 months ago

Known issue: If the caller crashes, the generator will leak.