labring / FastGPT

FastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing, RAG retrieval, and visual AI workflow orchestration, letting you easily develop and deploy complex question-answering systems without the need for extensive setup or configuration.
https://fastgpt.in
Other
16.93k stars 4.52k forks source link

无法使用本地视觉模型 #1308

Closed QichangZheng closed 4 months ago

QichangZheng commented 4 months ago

例行检查

你的版本 docker部署,已经确认所有镜像都是最新版

问题描述 问题和https://github.com/labring/FastGPT/issues/639 类似,但是我已经是最新版(4.7), oneapi也是最新版 API模型可以正常解析图片,例如gpt4-vision ollama部署的llava模型在webui调试,可以解析图片,所以模型没问题 在fastgpt可以和llava对话,说明模型正常加载,但是解析图片报错,如截图所示

复现步骤

预期结果

相关截图

cc1dfc2d3d9055f12c25d8c91afd466
nongmo677 commented 4 months ago

V4.7.1 测试无误

image

c121914yu commented 4 months ago

接口没对齐 gpt4v 把

QichangZheng commented 4 months ago

V4.7.1 测试无误

image

请问可以看一下你的配置信息吗?有可能是我配置错了

QichangZheng commented 4 months ago

接口没对齐 gpt4v 把

我不确定有没有对其,但是gpt4-v和Claude3在我的fastgpt是可以正常解析图片的。

nongmo677 commented 4 months ago

oneapi渠道选择 ollama渠道

QichangZheng commented 4 months ago

已解决: fastgpt需要指定v4.71版本,我默认latest是v4.7 oneapi的ollama渠道需要设置为ollama,而不是OpenAI