[bug] error with openai proxy and lang chain

[dtam]

Describe the bug should be able to chain.invoke via the OpenAI callable in langchain and a base url pointing at a guardrails server but it errors on role of type None

To Reproduce Steps to reproduce the behavior:

1. setup a guardrails server with 1 guard and 1 validator such as gibberish text
2. configure a langchain chain with a openai callable that has base_url set to the gr servers guards/guard_name/openai/v1 endpoint
3. invoke the chain syncronously

Expected behavior should complete and validate

Library version: 0.5.0

Additional context

[CalebCourier]

@dtam I think they originally were using ainvoke which might explain the behaviour. I believe this comment in the docs means that, for LCEL, ainvoke is equivalent to astream:

Whenever using runnables without LCEL, make sure to call .astream() on LLMs rather than .ainvoke to force the LLM to stream tokens.

Source: https://python.langchain.com/v0.1/docs/expression_language/interface/#async-stream-events-beta

This would explain why it worked when telling the model to activate streaming (that should send the stream=True keyword to the guardrails server). Still not sure why it blew up otherwise, but probably has something to do with the fact that Langchain was expecting a text/event-stream but was just getting application/json

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 14 days with no activity.

[CalebCourier]

I tested this again today to see where we were on being able to support this. The call actually goes through and validates appropriately but since we are not responding with the entire message structure including the role, Langchain's OpenAI pydantic model blows up.

Also, the streaming implementation does not work with ainvoke with or without setting streaming=True on the OpenAI model. It yields the following error in the API meaning we're not handling the structures correctly: data: {"error": {"message": "object generator can't be used in 'await' expression"}}