search

netease-youdao / QAnything

Question and Answer based on Anything.
https://qanything.ai
GNU Affero General Public License v3.0
11.86k stars 1.15k forks source link

[BUG] <title>纯python版调用ollama接口后台报找不到大模型 #425

Closed truthsun22 closed 4 months ago

truthsun22 commented 4 months ago

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

当前行为 | Current Behavior

纯python版，接口采用ollama方式，大模型用的qwen:7b。修改了run_for_openai_api_with_gpu_in_Linux_or_WSL.sh，修改内容如下：

!/bin/bash

bash scripts/base_run.sh -s "LinuxOrWSL" -w 4 -m 19530 -q 8777 -o -b 'http://localhost:11434/v1' -k 'ollama' -n 'qwen:7b' -l '4096'

期望行为 | Expected Behavior

No response

运行环境 | Environment

- OS:ubuntu
- NVIDIA Driver:
- CUDA:11.3
- docker:
- docker-compose:
- NVIDIA GPU:
- NVIDIA GPU Memory:

QAnything日志 | QAnything logs

2024-07-09 11:01:18,935 - [PID: 285561][MainProcess] - [Function: num_tokens_from_docs] - INFO - Warning: model not found. Using cl100k_base encoding. 2024-07-09 11:01:18,935 - [PID: 285561][MainProcess] - [Function: num_tokens_from_docs] - INFO - Warning: model not found. Using cl100k_base encoding. 2024-07-09 11:01:18,935 - [PID: 285561][MainProcess] - [Function: num_tokens_from_messages] - INFO - Warning: qwen:7b not found. Using cl100k_base encoding. 2024-07-09 11:01:18,935 - [PID: 285561][MainProcess] - [Function: num_tokens_from_messages] - INFO - Warning: qwen:7b may update over time. Returning num tokens assuming gpt-3.5-turbo-1106. 2024-07-09 11:01:18,935 - [PID: 285561][MainProcess] - [Function: num_tokens_from_messages] - INFO - Warning: qwen:7b not found. Using cl100k_base encoding. 2024-07-09 11:01:18,935 - [PID: 285561][MainProcess] - [Function: num_tokens_from_messages] - INFO - Warning: qwen:7b may update over time. Returning num tokens assuming gpt-3.5-turbo-1106. 2024-07-09 11:01:18,935 - [PID: 285561][MainProcess] - [Function: reprocess_source_documents] - INFO - limited token nums: 3290

复现方法 | Steps To Reproduce

No response

备注 | Anything else?

我环境中ollama已经安装了qwen:7b，但是日志里还是报找不到

simonjoe246 commented 4 months ago

@truthsun22 你好，请问问题是怎么解决的？我使用fastchat部署模型，也报这个错误

2024-07-11 09:11:46,341 Warning: model not found. Using cl100k_base encoding.
2024-07-11 09:11:46,342 Warning: model not found. Using cl100k_base encoding.
2024-07-11 09:11:46,342 Warning: model not found. Using cl100k_base encoding.
2024-07-11 09:11:46,343 Warning: model not found. Using cl100k_base encoding.
2024-07-11 09:11:46,343 Warning: model not found. Using cl100k_base encoding.
2024-07-11 09:11:46,344 limited token nums: 7434
2024-07-11 09:11:46,344 template token nums: 154
2024-07-11 09:11:46,344 query token nums: 21
2024-07-11 09:11:46,345 history token nums: 21
2024-07-11 09:11:46,345 Warning: model not found. Using cl100k_base encoding.
2024-07-11 09:11:46,348 new_source_docs token nums: 1991
2024-07-11 09:11:46,348 history_len: 2
2024-07-11 09:11:46,349 Warning: Qwen-7B-QAnything not found. Using cl100k_base encoding.
2024-07-11 09:11:46,349 Warning: Qwen-7B-QAnything may update over time. Returning num tokens assuming gpt-3.5-turbo-1106.
yangpeng666 commented 3 months ago

我也遇到了这个问题

jkongWPJK commented 3 months ago

我也是这个问题T.T

jkongWPJK commented 3 months ago

452 我解出了，可以看一下， 希望有帮助