guardrails-ai / guardrails

Adding guardrails to large language models.
https://www.guardrailsai.com/docs
Apache License 2.0
4.07k stars 310 forks source link

[bug] Warning on standard behavior #294

Closed zsimjee closed 1 year ago

zsimjee commented 1 year ago

Describe the bug Warning on default behavior in step 3 of the recipe generation notebook - 'Async event loop found, but guard was invoked synchronously.For validator parallelization, please call validate_async instead.'

To Reproduce Steps to reproduce the behavior:

  1. examples/recipe_generation.ipynb

Expected behavior Shouldn't warn on async/sync conflicts when using standard validators and railspecs

Library version: 0.2.0

Additional context Add any other context about the problem here.

aymsahdata commented 1 year ago

Also facing this issue when integrating with LangChain GuardRailsOutputParser to create an output parser and giving it the raw JSON answer of a faiss vectorstore guided LLM.

florence26 commented 1 year ago

Facing this issue on all examples docs and on version 0.2.1 as well.

irgolic commented 1 year ago

This warning is triggered whenever you're invoking guardrails synchronously in the presence of an async event loop.

Jupyter automatically creates an async eventloop, and lets you await things in notebook cells.

Try using this instead:

raw_llm_response, validated_response = await guard(
    openai.Completion.acreate, 
    engine="text-davinci-003", 
    max_tokens=2048, 
    temperature=0
)

Notice the use of await and openai.Completion.acreate instead of openai.Completion.create.

zsimjee commented 1 year ago

release will be out in pre-release 0.2.2-alpha1 today and as a patch deployment 0.2.3 next week