search

labring / FastGPT

FastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing, RAG retrieval, and visual AI workflow orchestration, letting you easily develop and deploy complex question-answering systems without the need for extensive setup or configuration.
https://tryfastgpt.ai
Other
17.49k stars 4.69k forks source link

与gemma:2b对接，相应非常慢 #1018

Closed taoshanghu closed 7 months ago

taoshanghu commented 7 months ago

环境： 系统：rocky:9 物理主机：40核 128G 400G ssd磁盘 独占

fastgpt: v4.6.8 one-api : 0.6.1 ollama: 0.1.27 m3e: registry.cn-hangzhou.aliyuncs.com/fastgpt_docker/m3e-large-api:latest LLM: gemma-2b

部署后 每次通过AI 提问 要50秒以上才能回复， 这还是我一个人操作，这个速度有没有优化的空间？或者是LLM不符

c121914yu commented 7 months ago

这个和 fastgpt 没关系呢。。模型问题得找模型本身。

没 gpu，跑模型不合适。