Routing Example Does Not Work with langchain-openai > 0.1.8

rabader commented 4 weeks ago

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

code from this link: https://python.langchain.com/v0.1/docs/use_cases/query_analysis/techniques/routing/

import getpass
import os

os.environ["OPENAI_API_KEY"] = getpass.getpass()

# Optional, uncomment to trace runs with LangSmith. Sign up here: https://smith.langchain.com.
# os.environ["LANGCHAIN_TRACING_V2"] = "true"
# os.environ["LANGCHAIN_API_KEY"] = getpass.getpass()

from typing import Literal

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_openai import ChatOpenAI

class RouteQuery(BaseModel):
    """Route a user query to the most relevant datasource."""

    datasource: Literal["python_docs", "js_docs", "golang_docs"] = Field(
        ...,
        description="Given a user question choose which datasource would be most relevant for answering their question",
    )

llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)
structured_llm = llm.with_structured_output(RouteQuery)

system = """You are an expert at routing a user question to the appropriate data source.

Based on the programming language the question is referring to, route it to the relevant data source."""
prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system),
        ("human", "{question}"),
    ]
)

router = prompt | structured_llm

question = """Why doesn't the following code work:

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages(["human", "speak in {language}"])
prompt.invoke("french")
"""
router.invoke({"question": question})

Error Message and Stack Trace (if applicable)

---------------------------------------------------------------------------

UnprocessableEntityError                  Traceback (most recent call last)

Cell In[6], line 8

      1 question = """Why doesn't the following code work:

      2

      3 from langchain_core.prompts import ChatPromptTemplate

   (...)

      6 prompt.invoke("french")

      7 """

----> 8 router.invoke({"question": question})

File c:\Users\MYUSERNAME\AppData\Local\miniconda3\envs\llm-local\lib\site-packages\langchain_core\runnables\base.py:2399, in RunnableSequence.invoke(self, input, config)

   2397 try:

   2398     for i, step in enumerate(self.steps):

-> 2399         input = step.invoke(

   2400             input,

   2401             # mark each step as a child run

   2402             patch_config(

   2403                 config, callbacks=run_manager.get_child(f"seq:step:{i+1}")

   2404             ),

   2405         )

   2406 # finish the root run

   2407 except BaseException as e:

File c:\Users\MYUSERNAME\AppData\Local\miniconda3\envs\llm-local\lib\site-packages\langchain_core\runnables\base.py:4433, in RunnableBindingBase.invoke(self, input, config, **kwargs)

   4427 def invoke(

   4428     self,

   4429     input: Input,

   4430     config: Optional[RunnableConfig] = None,

   4431     **kwargs: Optional[Any],

   4432 ) -> Output:

-> 4433     return self.bound.invoke(

   4434         input,

   4435         self._merge_configs(config),

   4436         **{**self.kwargs, **kwargs},

   4437     )

File c:\Users\MYUSERNAME\AppData\Local\miniconda3\envs\llm-local\lib\site-packages\langchain_core\language_models\chat_models.py:170, in BaseChatModel.invoke(self, input, config, stop, **kwargs)

    159 def invoke(

    160     self,

    161     input: LanguageModelInput,

   (...)

    165     **kwargs: Any,

    166 ) -> BaseMessage:

    167     config = ensure_config(config)

    168     return cast(

    169         ChatGeneration,

--> 170         self.generate_prompt(

    171             [self._convert_input(input)],

    172             stop=stop,

    173             callbacks=config.get("callbacks"),

    174             tags=config.get("tags"),

    175             metadata=config.get("metadata"),

    176             run_name=config.get("run_name"),

    177             run_id=config.pop("run_id", None),

    178             **kwargs,

    179         ).generations[0][0],

    180     ).message

File c:\Users\MYUSERNAME\AppData\Local\miniconda3\envs\llm-local\lib\site-packages\langchain_core\language_models\chat_models.py:599, in BaseChatModel.generate_prompt(self, prompts, stop, callbacks, **kwargs)

    591 def generate_prompt(

    592     self,

    593     prompts: List[PromptValue],

   (...)

    596     **kwargs: Any,

    597 ) -> LLMResult:

    598     prompt_messages = [p.to_messages() for p in prompts]

--> 599     return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)

File c:\Users\MYUSERNAME\AppData\Local\miniconda3\envs\llm-local\lib\site-packages\langchain_core\language_models\chat_models.py:456, in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, run_id, **kwargs)

    454         if run_managers:

    455             run_managers[i].on_llm_error(e, response=LLMResult(generations=[]))

--> 456         raise e

    457 flattened_outputs = [

    458     LLMResult(generations=[res.generations], llm_output=res.llm_output)  # type: ignore[list-item]

    459     for res in results

    460 ]

    461 llm_output = self._combine_llm_outputs([res.llm_output for res in results])

File c:\Users\MYUSERNAME\AppData\Local\miniconda3\envs\llm-local\lib\site-packages\langchain_core\language_models\chat_models.py:446, in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, run_id, **kwargs)

    443 for i, m in enumerate(messages):

    444     try:

    445         results.append(

--> 446             self._generate_with_cache(

    447                 m,

    448                 stop=stop,

    449                 run_manager=run_managers[i] if run_managers else None,

    450                 **kwargs,

    451             )

    452         )

    453     except BaseException as e:

    454         if run_managers:

File c:\Users\MYUSERNAME\AppData\Local\miniconda3\envs\llm-local\lib\site-packages\langchain_core\language_models\chat_models.py:671, in BaseChatModel._generate_with_cache(self, messages, stop, run_manager, **kwargs)

    669 else:

    670     if inspect.signature(self._generate).parameters.get("run_manager"):

--> 671         result = self._generate(

    672             messages, stop=stop, run_manager=run_manager, **kwargs

    673         )

    674     else:

    675         result = self._generate(messages, stop=stop, **kwargs)

File c:\Users\MYUSERNAME\AppData\Local\miniconda3\envs\llm-local\lib\site-packages\langchain_openai\chat_models\base.py:543, in BaseChatOpenAI._generate(self, messages, stop, run_manager, **kwargs)

    541 message_dicts, params = self._create_message_dicts(messages, stop)

    542 params = {**params, **kwargs}

--> 543 response = self.client.create(messages=message_dicts, **params)

    544 return self._create_chat_result(response)

File c:\Users\MYUSERNAME\AppData\Local\miniconda3\envs\llm-local\lib\site-packages\openai\_utils\_utils.py:277, in required_args.<locals>.inner.<locals>.wrapper(*args, **kwargs)

    275             msg = f"Missing required argument: {quote(missing[0])}"

    276     raise TypeError(msg)

--> 277 return func(*args, **kwargs)

File c:\Users\MYUSERNAME\AppData\Local\miniconda3\envs\llm-local\lib\site-packages\openai\resources\chat\completions.py:590, in Completions.create(self, messages, model, frequency_penalty, function_call, functions, logit_bias, logprobs, max_tokens, n, presence_penalty, response_format, seed, stop, stream, stream_options, temperature, tool_choice, tools, top_logprobs, top_p, user, extra_headers, extra_query, extra_body, timeout)

    558 @required_args(["messages", "model"], ["messages", "model", "stream"])

    559 def create(

    560     self,

   (...)

    588     timeout: float | httpx.Timeout | None | NotGiven = NOT_GIVEN,

    589 ) -> ChatCompletion | Stream[ChatCompletionChunk]:

--> 590     return self._post(

    591         "/chat/completions",

    592         body=maybe_transform(

    593             {

    594                 "messages": messages,

    595                 "model": model,

    596                 "frequency_penalty": frequency_penalty,

    597                 "function_call": function_call,

    598                 "functions": functions,

    599                 "logit_bias": logit_bias,

    600                 "logprobs": logprobs,

    601                 "max_tokens": max_tokens,

    602                 "n": n,

    603                 "presence_penalty": presence_penalty,

    604                 "response_format": response_format,

    605                 "seed": seed,

    606                 "stop": stop,

    607                 "stream": stream,

    608                 "stream_options": stream_options,

    609                 "temperature": temperature,

    610                 "tool_choice": tool_choice,

    611                 "tools": tools,

    612                 "top_logprobs": top_logprobs,

    613                 "top_p": top_p,

    614                 "user": user,

    615             },

    616             completion_create_params.CompletionCreateParams,

    617         ),

   618         options=make_request_options(

    619             extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout

    620         ),

    621         cast_to=ChatCompletion,

    622         stream=stream or False,

    623         stream_cls=Stream[ChatCompletionChunk],

    624     )

File c:\Users\MYUSERNAME\AppData\Local\miniconda3\envs\llm-local\lib\site-packages\openai\_base_client.py:1240, in SyncAPIClient.post(self, path, cast_to, body, options, files, stream, stream_cls)

   1226 def post(

   1227     self,

   1228     path: str,

   (...)

   1235     stream_cls: type[_StreamT] | None = None,

   1236 ) -> ResponseT | _StreamT:

   1237     opts = FinalRequestOptions.construct(

   1238         method="post", url=path, json_data=body, files=to_httpx_files(files), **options

   1239     )

-> 1240     return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))

File c:\Users\MYUSERNAME\AppData\Local\miniconda3\envs\llm-local\lib\site-packages\openai\_base_client.py:921, in SyncAPIClient.request(self, cast_to, options, remaining_retries, stream, stream_cls)

    912 def request(

    913     self,

    914     cast_to: Type[ResponseT],

   (...)

    919     stream_cls: type[_StreamT] | None = None,

    920 ) -> ResponseT | _StreamT:

--> 921     return self._request(

    922         cast_to=cast_to,

    923         options=options,

    924         stream=stream,

    925         stream_cls=stream_cls,

    926         remaining_retries=remaining_retries,

    927     )

File c:\Users\MYUSERNAME\AppData\Local\miniconda3\envs\llm-local\lib\site-packages\openai\_base_client.py:1020, in SyncAPIClient._request(self, cast_to, options, remaining_retries, stream, stream_cls)

   1017         err.response.read()

   1019     log.debug("Re-raising status error")

-> 1020     raise self._make_status_error_from_response(err.response) from None

   1022 return self._process_response(

   1023     cast_to=cast_to,

   1024     options=options,

   (...)

   1027     stream_cls=stream_cls,

   1028 )

UnprocessableEntityError: Error code: 422 - {'detail': [{'type': 'enum', 'loc': ['body', 'tool_choice', 'str-enum[ChatCompletionToolChoiceOptionEnum]'], 'msg': "Input should be 'none' or 'auto'", 'input': 'required', 'ctx': {'expected': "'none' or 'auto'"}}, {'type': 'model_attributes_type', 'loc': ['body', 'tool_choice', 'ChatCompletionNamedToolChoice'], 'msg': 'Input should be a valid dictionary or object to extract fields from', 'input': 'required'}]}

Description

When using the routing example shown in the langchain docs, it only works if the "langchain-openai" version is 0.1.8 or lower. The newest versions (0.1.9+) break this logic. Routers are used in my workflow and this is preventing me from upgrading my packages. Please either revert the breaking changes or provide new documentation to support this type of routing functionality.

System Info

langchain ==0.2.3 langchain-chroma ==0.1.1 langchain-community ==0.2.0 langchain-core ==0.2.3 langchain-experimental ==0.0.59 langchain-google-genai ==1.0.4 langchain-google-vertexai ==1.0.4 langchain-openai ==0.1.10 langchain-text-splitters ==0.2.0 langchainhub ==0.1.15 langgraph ==0.1.1 openai ==1.27.0

platform: windows python version 3.10.10

ccurme commented 4 weeks ago

Hello, thanks for this. Would you mind sharing what version of the openai Python sdk you are using?

rabader commented 4 weeks ago

Hello, thanks for this. Would you mind sharing what version of the openai Python sdk you are using?

Just added that above. It's 1.27.0

rabader commented 4 weeks ago

I'll also add that I just tried running this with the latest version of the openai SDK (1.35.4) and got the same error.

baskaryan commented 4 weeks ago

Are you sure you've updated the sdk in the correct python environment (and refreshed your jupyter kernel if you're in a notebook)? "required" is definitely a supported tool_choice value in the latest sdk https://github.com/openai/openai-python/blob/main/src/openai/types/chat/chat_completion_tool_choice_option_param.py

rabader commented 3 weeks ago

Are you sure you've updated the sdk in the correct python environment (and refreshed your jupyter kernel if you're in a notebook)? "required" is definitely a supported tool_choice value in the latest sdk https://github.com/openai/openai-python/blob/main/src/openai/types/chat/chat_completion_tool_choice_option_param.py

Hmmm, I've been using my enterptise' internal OpenAI-compatible API, but looking at their code, it looks like they're currently working off of openai==1.10.0. I guess that's the problem, regardless of what I have installed in my env.

langchain-ai / langchain