xorbitsai / inference

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
https://inference.readthedocs.io
Apache License 2.0
4.68k stars 367 forks source link

使用langchain的agent调用时,返回400错误 #1235

Closed lordk911 closed 3 weeks ago

lordk911 commented 5 months ago

Describe the bug

代码实例:https://python.langchain.com/docs/use_cases/sql/agents

To Reproduce

To help us to reproduce this bug, please provide information below:

  1. Your Python version. 3.10
  2. The version of xinference you use. 0.10.0
  3. Versions of crucial packages.
    client conda env : 
        langchain                0.1.14
        langchain-community      0.0.30
        langchain-core           0.1.37
        langchain-experimental   0.0.56
        langchain-openai         0.1.1
        langchain-text-splitters 0.0.1
        langsmith                0.1.38
        openai                   1.14.3
  1. Full stack of the error.
Traceback (most recent call last):
  File "/Users/shin/ideaProjects/langchain_learn/chat-sql.py", line 37, in <module>
    agent_executor.invoke("List the total sales per country. Which country's customers spent the most?")
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/langchain/chains/base.py", line 163, in invoke
    raise e
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/langchain/chains/base.py", line 153, in invoke
    self._call(inputs, run_manager=run_manager)
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/langchain/agents/agent.py", line 1432, in _call
    next_step_output = self._take_next_step(
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/langchain/agents/agent.py", line 1138, in _take_next_step
    [
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/langchain/agents/agent.py", line 1138, in <listcomp>
    [
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/langchain/agents/agent.py", line 1166, in _iter_next_step
    output = self.agent.plan(
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/langchain/agents/agent.py", line 514, in plan
    for chunk in self.runnable.stream(inputs, config={"callbacks": callbacks}):
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 2875, in stream
    yield from self.transform(iter([input]), config, **kwargs)
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 2862, in transform
    yield from self._transform_stream_with_config(
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 1880, in _transform_stream_with_config
    chunk: Output = context.run(next, iterator)  # type: ignore
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 2826, in _transform
    for output in final_pipeline:
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 1283, in transform
    for chunk in input:
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 4722, in transform
    yield from self.bound.transform(
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 1300, in transform
    yield from self.stream(final, config, **kwargs)
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 241, in stream
    raise e
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 223, in stream
    for chunk in self._stream(messages, stop=stop, **kwargs):
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/langchain_openai/chat_models/base.py", line 441, in _stream
    for chunk in self.client.create(messages=message_dicts, **params):
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/openai/_utils/_utils.py", line 275, in wrapper
    return func(*args, **kwargs)
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/openai/resources/chat/completions.py", line 667, in create
    return self._post(
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/openai/_base_client.py", line 1208, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/openai/_base_client.py", line 897, in request
    return self._request(
  File "/Users/shin/miniconda3/envs/langchain_official/lib/python3.10/site-packages/openai/_base_client.py", line 988, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'detail': 'Invalid input. Please specify the prompt.'}
  1. Minimized code to reproduce the error.
from langchain_community.agent_toolkits import create_sql_agent
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model="qwen1.5-chat", temperature=0, api_key="", base_url=f"http://10.9.123.466:9997/v1")
agent_executor = create_sql_agent(llm, db=db, agent_type="openai-tools", verbose=True)
agent_executor.invoke("List the total sales per country. Which country's customers spent the most?")

Expected behavior

A clear and concise description of what you expected to happen.

Additional context

调用api时的参数:

image
zhangzhuoran997997 commented 3 months ago

I encountered the same problem, do you have any better solution now?

github-actions[bot] commented 1 month ago

This issue is stale because it has been open for 7 days with no activity.

github-actions[bot] commented 3 weeks ago

This issue was closed because it has been inactive for 5 days since being marked as stale.