modelscope / agentscope

Start building LLM-empowered multi-agent applications in an easier way.
https://doc.agentscope.io/
Apache License 2.0
5.36k stars 330 forks source link

[Bug]:使用qwen-vl-plus模型报错400错误 #88

Closed Haijian06 closed 8 months ago

Haijian06 commented 8 months ago

Describe` the bug 当我在模型配置中使用了dashscope api的识图模型qwen-vl-plus,在使用Msg模块传入图片给agent时报错400,url换过本地和网络,都是报错400。 To Reproduce Steps to reproduce the behavior:

  1. You code
import agentscope
from agentscope.agents import DialogAgent,UserAgent
from agentscope.message import Msg
from agentscope.pipelines.functional import sequentialpipeline
agentscope.init(model_configs=[
    {
        "config_name": "qwen-vl-plus",
        "model_type": "dashscope_text_embedding",
        "model_name": "qwen-vl-plus",
        "api_key": "sk-df---314ae----------------------" 
    }
])

agent_1 = DialogAgent(
    name="助手",
    sys_prompt="你是一位好帮手",
    model_config_name="qwen-vl-plus"
)
useragent = UserAgent()
x = Msg(name='用户',content='帮我描述一下这个图片',url="https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg")
print(x)
while x is None or x.content != "exit":
  x = sequentialpipeline([agent_1,useragent])
  1. How to execute

    我是从本地安装的

    git clone https://github.com/modelscope/agentscope.git
    cd AgentScope
    pip install -e .
  2. See error

Traceback (most recent call last):
  File "D:\python_project_yeah\NLP\agent\7agent处理图像测试.py", line 23, in <module>
    x = sequentialpipeline([agent_1,useragent])
  File "D:\python_project_yeah\NLP\agentscope\src\agentscope\pipelines\functional.py", line 44, in sequentialpipeline
    msg = operators[0](x)
  File "D:\python_project_yeah\NLP\agentscope\src\agentscope\agents\agent.py", line 117, in __call__
    res = self.reply(*args, **kwargs)
  File "D:\python_project_yeah\NLP\agentscope\src\agentscope\agents\dialog_agent.py", line 81, in reply
    response = self.model(prompt).text
  File "D:\python_project_yeah\NLP\agentscope\src\agentscope\models\model.py", line 166, in checking_wrapper
    return model_call(self, *args, **kwargs)
  File "D:\python_project_yeah\NLP\agentscope\src\agentscope\models\dashscope_model.py", line 412, in __call__
    raise RuntimeError(error_msg)
RuntimeError:  Request id: 4b565370-9acd-9eda-8a6c-94668a1fe8b1, Status code: 400, error code: InvalidParameter, error message: url error, please check url!.

Expected behavior 成功运行

Environment (please complete the following information):

qbc2016 commented 8 months ago

There seems to be a misunderstanding regarding the functionality of our current implementation of the DashScope wrapper. At present, it includes capabilities for chat, image synthesis, and text embedding but does not extend to multimodal interactions. The error you've encountered stems from the specified "model_type": "dashscope_text_embedding" within your model_configs. This configuration explicitly invokes the dashscope.TextEmbedding.call method, which unfortunately does not support the "model_name": "qwen-vl-plus" as indicated by the URL error message.

If your intention is to utilize dashscope.MultiModalConversation.call, I shall inform you that this interface is not yet implemented in our system. Maybe we will implement it later. However, if you're keen on accessing this functionality, you are welcome to develop a DashScopeMultiModalWrapper by following the existing implementation patterns found in the dashscope_model.py. For reference on the parameters required, please visit https://help.aliyun.com/zh/dashscope/developer-reference/tongyi-qianwen-vl-plus-api.

We would also be delighted to consider any pull requests to our library should you decide to contribute your implementation.

Haijian06 commented 8 months ago

好的谢谢你的解答,那根据我看到在目前的AgentScope中,是不是还不支持agent多模态推理功能。

model

如果支持那有具体的文档步骤吗?谢谢

qbc2016 commented 8 months ago

Currently, we do not have support for multimodal reasoning functionality, but we plan to continuously extend our support for additional models. If multimodal reasoning capabilities are essential for your needs, we highly encourage you to take the initiative to implement a DashScopeMultiModalWrapper. For guidance, you can refer to the implementation of the other wrappers found within dashscope_model.py. Your contributions are valuable, and we warmly invite you to submit pull requests to our repository.

Haijian06 commented 8 months ago

好的谢谢,我会去试试的,感谢你的解答!