solve customized OpenAIVision model api_host issue

[vc12345679]

作者自述

描述

更正 OpenAIVision 内自定义 api_host 的调用。

更正后，以 DeepSeek V2 模型为例，相应 json 配置如下

"extra_model_metadata": {
      "DeepSeek V2": {
            "model_name": "deepseek-chat",
            "description": "This is DeepSeek's powerful open source LLM.",
            "model_type": "OpenAIVision",
            "multimodal": false,
            "api_host": "https://api.deepseek.com",
            "api_key": "sk-xxxxxxxx",
            "token_limit": 4096,
            "system": "You are a helpful AI assistant.",
            "placeholder": {
                "logo": "https://avatars.githubusercontent.com/u/148330874",
            }
      },
},

运行结果

相关问题

1117

补充信息

OpenAIVisionClient.__init__() 时已进行自定义 api_host 判断，并正确赋值给一系列 self.xxx_url，如下：

        if self.api_host is not None:
            self.chat_completion_url, self.images_completion_url, self.openai_api_base, self.balance_api_url, self.usage_api_url = shared.format_openai_host(self.api_host)
        else:
            self.api_host, self.chat_completion_url, self.images_completion_url, self.openai_api_base, self.balance_api_url, self.usage_api_url = shared.state.api_host, shared.state.chat_completion_url, shared.state.images_completion_url, shared.state.openai_api_base, shared.state.balance_api_url, shared.state.usage_api_url

此时各变量分别为

self.api_host               https://api.deepseek.com
self.chat_completion_url        https://api.deepseek.com/v1/chat/completions
self.images_completion_url      https://api.deepseek.com/v1/images/generations
self.openai_api_base            https://api.deepseek.com/v1
self.balance_api_url            https://api.deepseek.com/dashboard/billing/credit_grants
self.usage_api_url          https://api.deepseek.com/dashboard/billing/usage

但在_get_response()和_single_query_at_once()中，执行自定义api_host判断时

        # 如果有自定义的api-host，使用自定义host发送请求，否则使用默认设置发送请求
        if shared.state.chat_completion_url != CHAT_COMPLETION_URL:
            logging.debug(f"使用自定义API URL: {shared.state.chat_completion_url}")

各变量分别为

shared.state.chat_completion_url    https://api.openai.com/v1/chat/completions
CHAT_COMPLETION_URL         https://api.openai.com/v1/chat/completions

因此，相应的 requests.post 所对应的 url 应为 self.chat_completion_url，而非 shared.state.chat_completion_url