BerriAI / reliableGPT

Get 100% uptime, reliability from OpenAI. Handle Rate Limit, Timeout, API, Keys Errors
MIT License
630 stars 37 forks source link

Acreate #36

Open bballboy8 opened 1 year ago

bballboy8 commented 1 year ago

When will there be support for asynchronous usage?

krrishdholakia commented 1 year ago

hey @bballboy8 why do you need acreate support?

i've seen people make async calls and just await the openai response inside the function, so i'm not sure i understand the use-case

bballboy8 commented 1 year ago

So for something like this, it would make more sense to have an async call within the async function:

async def _call_gpt(prompt: str, **kwargs: Any) -> Dict:
    """Call GPT api with backoff."""
    openai.ChatCompletion.acreate = reliableGPT(
        openai.ChatCompletion.acreate, user_email="charlie@reclip.app"
    )
    params = {**DEFAULT_GPT_PARAMS, **kwargs}
    return await openai.ChatCompletion.acreate(
        model=settings.OPENAI_GPT_MODEL,
        messages=[dict(role="user", content=prompt)],
        request_timeout=15,
        **params,
    )