search

labring / FastGPT

FastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing, RAG retrieval, and visual AI workflow orchestration, letting you easily develop and deploy complex question-answering systems without the need for extensive setup or configuration.
https://tryfastgpt.ai
Other
17.45k stars 4.69k forks source link

fastgpt调用ollama或者xinference时config.json中的maxContent是否生效？ #2780

Closed SDAIer closed 3 weeks ago

SDAIer commented 3 weeks ago

4.8.10-fx oneapi 0.6.7

fastgpt调用ollama或者xinference时，config.json中的maxContent都不生效吗？？

xinference log

image image

ollama log

image image

SDAIer commented 3 weeks ago

@c121914yu

c121914yu commented 3 weeks ago

maxContext 是上下文，maxResponse 是max_tokens，然后你还要看 oneapi 支不支持参数转化

SDAIer commented 3 weeks ago

maxContext 是上下文，maxResponse 是max_tokens，然后你还要看 oneapi 支不支持参数转化

oneapi那边反馈如下: 这个直接传num_ctx参数就可以了，用0.6.9alpha版本，我加了这个参数进去.docker-compose.yml里image那里制定一下版本，然后怎么传这个参数你得看fastgpt的文档

@c121914yu 麻烦fastgpt增加下这个参数，多谢

通过curl测试新增加的参数num_ctx的确可以 curl --location --request POST 'http://172.22.1.39:3001/v1/chat/completions' \ --header 'Authorization: Bearer sk-iYrST123' \ --header 'Content-Type: application/json' \ --data '{ "model": "qwen2:latest", "stream": false, "temperature": 1, "max_tokens": 2000,
"num_ctx":7777, "messages": [ { "role": "user", "content": "你是谁" } ] }'

image