search

eosphoros-ai / DB-GPT

AI Native Data App Development framework with AWEL(Agentic Workflow Expression Language) and Agents
http://docs.dbgpt.cn
MIT License
13.65k stars 1.83k forks source link

[Bug][1211]use zhipuai proxy llm error #1093

Closed TaoPeiLing closed 9 months ago

TaoPeiLing commented 9 months ago

根据安装文档，通过源码部署，成功启动DB-GPT。但是开启问话时，就提示了“模型不存在，请检查模型代码”的问题。是否和我下载的pip install zhipuai的路径有关？我是在Db-gpt的主目录中执行pip install zhipuai的。不知道是否可以帮忙指导下。

[1211][模型不存在，请检查模型代码。]2024-01-19 21:24:42 DB-GPT dbgpt.model.cluster.worker.default_worker[6101] INFO is_first_generate, usage: None

full stream output: [1211][模型不存在，请检查模型代码。]

model generate_stream params: {'model': 'zhipu_proxyllm', 'messages': [ModelMessage(role='human', content='你好', round_index=1), ModelMessage(role='ai', content='[1211][模型不存在，请检查模型代码。] (error_code: 1)', round_index=1), ModelMessage(role='human', content='你好', round_index=2), ModelMessage(role='ai', content='[1211][模型不存在，请检查模型代码。] (error_code: 1)', round_index=2), ModelMessage(role='human', content='hi', round_index=3), ModelMessage(role='ai', content='[1211][模型不存在，请检查模型代码。] (error_code: 1)', round_index=3), ModelMessage(role='human', content='hi', round_index=4), ModelMessage(role='ai', content='[1211][模型不存在，请检查模型代码。] (error_code: 1)', round_index=4), ModelMessage(role='human', content='你是一个有用的 AI 助手。\nhi', round_index=0)], 'temperature': 0.6, 'max_new_tokens': 1024, 'echo': False, 'span_id': '32146930-31ba-4a01-8302-fff430878390:1d31155a-20c9-4bba-b05d-323b2cfac4d9', 'context': {'stream': True, 'cache_enable': False, 'user_name': None, 'sys_code': None, 'conv_uid': None, 'span_id': '32146930-31ba-4a01-8302-fff430878390:6d434afd-ced5-407c-a183-ff4a335ae88f', 'chat_mode': 'chat_normal', 'extra': {}, 'request_id': None}, 'convert_to_compatible_format': False, 'string_prompt': 'human: 你好\nai: [1211][模型不存在，请检查模型代码。] (error_code: 1)\nhuman: 你好\nai: [1211][模型不存在，请检查模型代码。] (error_code: 1)\nhuman: hi\nai: [1211][模型不存在，请检查模型代码。] (error_code: 1)\nhuman: hi\nai: [1211][模型不存在，请检查模型代码。] (error_code: 1)\nhuman: 你是一个有用的 AI 助手。\nhi'}

Aries-ckt commented 9 months ago

can you show your .env setting about llm ?

TaoPeiLing commented 9 months ago

感谢你的回复。下面是我的环境设置。`#***#

DB-GPT - GENERAL SETTINGS

***

DISABLED_COMMAND_CATEGORIES - The list of categories of commands that are disabled. Each of the below are an option:

pilot.commands.query_execute

For example, to disable coding related features, uncomment the next line

DISABLED_COMMAND_CATEGORIES=

***

Webserver Port

***

WEB_SERVER_PORT=7860

***

LLM PROVIDER

***

TEMPERATURE=0

***

LLM MODELS

***

LLM_MODEL, see dbgpt/configs/model_config.LLM_MODEL_CONFIG

LLM_MODEL=zhipu_proxyllm PROXY_SERVER_URL=https://open.bigmodel.cn/api/paas/v4 ZHIPU_MODEL_VERSION=v4 ZHIPU_PROXY_API_KEY=0e843c5ff6598a4d3bcdc3baddc80a4b.qNoZbSysGuNe1L8j

LLM model path, by default, DB-GPT will read the model path from LLM_MODEL_CONFIG based on the LLM_MODEL.

Of course you can specify your model path according to LLM_MODEL_PATH

In DB-GPT, the priority from high to low to read model path:

1. environment variable with key: {LLM_MODEL}_MODEL_PATH (Avoid multi-model conflicts)

2. environment variable with key: MODEL_PATH

3. environment variable with key: LLM_MODEL_PATH

4. the config in dbgpt/configs/model_config.LLM_MODEL_CONFIG

LLM_MODEL_PATH=/app/models/vicuna-13b-v1.5

LLM_PROMPT_TEMPLATE=vicuna_v1.1

MODEL_SERVER=http://127.0.0.1:8000 LIMIT_MODEL_CONCURRENCY=5 MAX_POSITION_EMBEDDINGS=4096 QUANTIZE_QLORA=True QUANTIZE_8bit=True

QUANTIZE_4bit=False

SMART_LLM_MODEL - Smart language model (Default: vicuna-13b)

FAST_LLM_MODEL - Fast language model (Default: chatglm-6b)

SMART_LLM_MODEL=vicuna-13b

FAST_LLM_MODEL=chatglm-6b

Proxy llm backend, this configuration is only valid when "LLM_MODEL=proxyllm", When we use the rest API provided by deployment frameworks like fastchat as a proxyllm,

"PROXYLLM_BACKEND" is the model they actually deploy. We can use "PROXYLLM_BACKEND" to load the prompt of the corresponding scene.

PROXYLLM_BACKEND=

You can configure parameters for a specific model with {model name}_{config key}=xxx

See dbgpt/model/parameter.py

prompt template for current model

llama_cpp_prompt_template=vicuna_v1.1

llama-2-70b must be 8

llama_cpp_n_gqa=8

Model path

llama_cpp_model_path=/data/models/TheBloke/vicuna-13B-v1.5-GGUF/vicuna-13b-v1.5.Q4_K_M.gguf

LLM cache

Enable Model cache

MODEL_CACHE_ENABLE=True

The storage type of model cache, now supports: memory, disk

MODEL_CACHE_STORAGE_TYPE=disk

The max cache data in memory, we always store cache data in memory fist for high speed.

MODEL_CACHE_MAX_MEMORY_MB=256

The dir to save cache data, this configuration is only valid when MODEL_CACHE_STORAGE_TYPE=disk

The default dir is pilot/data/model_cache

MODEL_CACHE_STORAGE_DISK_DIR=

***

EMBEDDING SETTINGS

***

EMBEDDING_MODEL=text2vec

EMBEDDING_MODEL=m3e-large

EMBEDDING_MODEL=bge-large-en

EMBEDDING_MODEL=bge-large-zh

KNOWLEDGE_CHUNK_SIZE=500 KNOWLEDGE_SEARCH_TOP_SIZE=5

KNOWLEDGE_CHUNK_OVERLAP=50

Control whether to display the source document of knowledge on the front end.

KNOWLEDGE_CHAT_SHOW_RELATIONS=False

Whether to enable Chat Knowledge Search Rewrite Mode

KNOWLEDGE_SEARCH_REWRITE=False

EMBEDDING_TOKENIZER - Tokenizer to use for chunking large inputs

EMBEDDING_TOKEN_LIMIT - Chunk size limit for large inputs

EMBEDDING_MODEL=all-MiniLM-L6-v2

EMBEDDING_TOKENIZER=all-MiniLM-L6-v2

EMBEDDING_TOKEN_LIMIT=8191

Openai embedding model, See dbgpt/model/parameter.py

EMBEDDING_MODEL=proxy_openai

proxy_openai_proxy_server_url=https://api.openai.com/v1

proxy_openai_proxy_api_key={your-openai-sk}

proxy_openai_proxy_backend=text-embedding-ada-002

***

DB-GPT METADATA DATABASE SETTINGS

***

SQLite database (Current default database)

LOCAL_DB_TYPE=sqlite

MYSQL database

LOCAL_DB_TYPE=mysql

LOCAL_DB_USER=root

LOCAL_DB_PASSWORD={your_password}

LOCAL_DB_HOST=127.0.0.1

LOCAL_DB_PORT=3306

LOCAL_DB_NAME=dbgpt

This option determines the storage location of conversation records. The default is not configured to the old version of duckdb. It can be optionally db or file (if the value is db, the database configured by LOCAL_DB will be used)

CHAT_HISTORY_STORE_TYPE=db

***

COMMANDS

***

EXECUTE_LOCAL_COMMANDS=False

***

ALLOWLISTED PLUGINS

***

ALLOWLISTED_PLUGINS - Sets the listed plugins that are allowed (Example: plugin1,plugin2,plugin3)

DENYLISTED_PLUGINS - Sets the listed plugins that are not allowed (Example: plugin1,plugin2,plugin3)

ALLOWLISTED_PLUGINS= DENYLISTED_PLUGINS=

***

CHAT PLUGIN SETTINGS

***

CHAT_MESSAGES_ENABLED - Enable chat messages (Default: False)

CHAT_MESSAGES_ENABLED=False

***

VECTOR STORE SETTINGS

***

Chroma vector db config

VECTOR_STORE_TYPE=Chroma

CHROMA_PERSIST_PATH=/root/DB-GPT/pilot/data

Milvus vector db config

VECTOR_STORE_TYPE=Milvus

MILVUS_URL=127.0.0.1

MILVUS_PORT=19530

MILVUS_USERNAME

MILVUS_PASSWORD

MILVUS_SECURE=

Weaviate vector db config

VECTOR_STORE_TYPE=Weaviate

WEAVIATE_URL=https://kt-region-m8hcy0wc.weaviate.network

***

WebServer Language Support

***

LANGUAGE=en

LANGUAGE=zh

***

** PROXY_SERVER (openai interface | chatGPT proxy service), use chatGPT as your LLM.

** if your server can visit openai, please set PROXY_SERVER_URL=https://api.openai.com/v1/chat/completions

** else if you have a chatgpt proxy server, you can set PROXY_SERVER_URL={your-proxy-serverip:port/xxx}

***

PROXY_API_KEY={your-openai-sk} PROXY_SERVER_URL=https://api.openai.com/v1/chat/completions

from https://bard.google.com/ f12-> application-> __Secure-1PSID

BARD_PROXY_API_KEY={your-bard-token}

***

PROXY_SERVER +

***

Aliyun tongyi

TONGYI_PROXY_API_KEY={your-tongyi-sk}

Baidu wenxin

WEN_XIN_MODEL_VERSION={version}

WEN_XIN_API_KEY={your-wenxin-sk}

WEN_XIN_API_SECRET={your-wenxin-sct}

Zhipu

ZHIPU_MODEL_VERSION={version}

ZHIPU_PROXY_API_KEY={your-zhipu-sk}

Baichuan

BAICHUN_MODEL_NAME={version}

BAICHUAN_PROXY_API_KEY={your-baichuan-sk}

BAICHUAN_PROXY_API_SECRET={your-baichuan-sct}

Xunfei Spark

XUNFEI_SPARK_API_VERSION={version}

XUNFEI_SPARK_APPID={your_app_id}

XUNFEI_SPARK_API_KEY={your_api_key}

XUNFEI_SPARK_API_SECRET={your_api_secret}

***

SUMMARY_CONFIG

***

SUMMARY_CONFIG=FAST

***

MUlti-GPU

***

See https://developer.nvidia.com/blog/cuda-pro-tip-control-gpu-visibility-cuda_visible_devices/

If CUDA_VISIBLE_DEVICES is not configured, all available gpus will be used

CUDA_VISIBLE_DEVICES=0

You can configure the maximum memory used by each GPU.

MAX_GPU_MEMORY=16Gib

***

LOG

***

FATAL, ERROR, WARNING, WARNING, INFO, DEBUG, NOTSET

DBGPT_LOG_LEVEL=INFO

LOG dir, default: ./logs

DBGPT_LOG_DIR=`

TaoPeiLing commented 9 months ago

打扰您了，十分抱歉。关于我的模型不存在的问题，我的env文件配置如下：

***

             DB-GPT  - GENERAL SETTINGS                        #  

***

DISABLED_COMMAND_CATEGORIES - The list of categories of commands that are disabled. Each of the below are an option:

pilot.commands.query_execute

For example, to disable coding related features, uncomment the next line

DISABLED_COMMAND_CATEGORIES=   

***

                        Webserver Port                         

***

WEB_SERVER_PORT=7860

***

                       LLM PROVIDER                         

***

TEMPERATURE=0

***

                         LLM MODELS                           

***

LLM_MODEL, see dbgpt/configs/model_config.LLM_MODEL_CONFIG

LLM_MODEL=zhipu_proxyllm

LLM model path, by default, DB-GPT will read the model path from LLM_MODEL_CONFIG based on the LLM_MODEL.

Of course you can specify your model path according to LLM_MODEL_PATH

In DB-GPT, the priority from high to low to read model path:

    1. environment variable with key: {LLM_MODEL}_MODEL_PATH (Avoid multi-model conflicts)

    2. environment variable with key: MODEL_PATH

    3. environment variable with key: LLM_MODEL_PATH

    4. the config in dbgpt/configs/model_config.LLM_MODEL_CONFIG

LLM_MODEL_PATH=/usr/local/miniconda3/lib/python3.11/site-packages/zhipuai

LLM_PROMPT_TEMPLATE=vicuna_v1.1

MODEL_SERVER=http://127.0.0.1:8000 LIMIT_MODEL_CONCURRENCY=5 MAX_POSITION_EMBEDDINGS=4096 QUANTIZE_QLORA=True QUANTIZE_8bit=True

QUANTIZE_4bit=False

SMART_LLM_MODEL - Smart language model (Default: vicuna-13b)

FAST_LLM_MODEL - Fast language model (Default: chatglm-6b)

SMART_LLM_MODEL=vicuna-13b

FAST_LLM_MODEL=chatglm-6b

Proxy llm backend, this configuration is only valid when "LLM_MODEL=proxyllm", When we use the rest API provided by deployment frameworks like fastchat as a proxyllm, 

"PROXYLLM_BACKEND" is the model they actually deploy. We can use "PROXYLLM_BACKEND" to load the prompt of the corresponding scene. 

PROXYLLM_BACKEND=

You can configure parameters for a specific model with {model name}_{config key}=xxx

See dbgpt/model/parameter.py

prompt template for current model

llama_cpp_prompt_template=vicuna_v1.1

llama-2-70b must be 8

llama_cpp_n_gqa=8

Model path

llama_cpp_model_path=/data/models/TheBloke/vicuna-13B-v1.5-GGUF/vicuna-13b-v1.5.Q4_K_M.gguf

LLM cache

Enable Model cache

MODEL_CACHE_ENABLE=True

The storage type of model cache, now supports: memory, disk

MODEL_CACHE_STORAGE_TYPE=disk

The max cache data in memory, we always store cache data in memory fist for high speed. 

MODEL_CACHE_MAX_MEMORY_MB=256

The dir to save cache data, this configuration is only valid when MODEL_CACHE_STORAGE_TYPE=disk

The default dir is pilot/data/model_cache

MODEL_CACHE_STORAGE_DISK_DIR=

***

                         EMBEDDING SETTINGS                   

***

EMBEDDING_MODEL=text2vec

EMBEDDING_MODEL=m3e-large

EMBEDDING_MODEL=bge-large-en

EMBEDDING_MODEL=bge-large-zh

KNOWLEDGE_CHUNK_SIZE=500 KNOWLEDGE_SEARCH_TOP_SIZE=5

KNOWLEDGE_CHUNK_OVERLAP=50

Control whether to display the source document of knowledge on the front end.

KNOWLEDGE_CHAT_SHOW_RELATIONS=False

Whether to enable Chat Knowledge Search Rewrite Mode

KNOWLEDGE_SEARCH_REWRITE=False

EMBEDDING_TOKENIZER   - Tokenizer to use for chunking large inputs

EMBEDDING_TOKEN_LIMIT - Chunk size limit for large inputs

EMBEDDING_MODEL=all-MiniLM-L6-v2

EMBEDDING_TOKENIZER=all-MiniLM-L6-v2

EMBEDDING_TOKEN_LIMIT=8191

Openai embedding model, See dbgpt/model/parameter.py

EMBEDDING_MODEL=proxy_openai

proxy_openai_proxy_server_url=https://api.openai.com/v1

proxy_openai_proxy_api_key={your-openai-sk}

proxy_openai_proxy_backend=text-embedding-ada-002

***

                  DB-GPT METADATA DATABASE SETTINGS           

***

SQLite database (Current default database)

LOCAL_DB_TYPE=sqlite

MYSQL database

LOCAL_DB_TYPE=mysql

LOCAL_DB_USER=root

LOCAL_DB_PASSWORD={your_password}

LOCAL_DB_HOST=127.0.0.1

LOCAL_DB_PORT=3306

LOCAL_DB_NAME=dbgpt

This option determines the storage location of conversation records. The default is not configured to the old version of duckdb. It can be optionally db or file (if the value is db, the database configured by LOCAL_DB will be used)

CHAT_HISTORY_STORE_TYPE=db

***

                         COMMANDS                             

***

EXECUTE_LOCAL_COMMANDS=False

***

                  ALLOWLISTED PLUGINS                         

***

ALLOWLISTED_PLUGINS - Sets the listed plugins that are allowed (Example: plugin1,plugin2,plugin3)

DENYLISTED_PLUGINS - Sets the listed plugins that are not allowed (Example: plugin1,plugin2,plugin3)

ALLOWLISTED_PLUGINS= DENYLISTED_PLUGINS=

***

                 CHAT PLUGIN SETTINGS                         

***

CHAT_MESSAGES_ENABLED - Enable chat messages (Default: False)

CHAT_MESSAGES_ENABLED=False

***

                  VECTOR STORE SETTINGS                       

***

Chroma vector db config

VECTOR_STORE_TYPE=Chroma

CHROMA_PERSIST_PATH=/root/DB-GPT/pilot/data

Milvus vector db config

VECTOR_STORE_TYPE=Milvus

MILVUS_URL=127.0.0.1

MILVUS_PORT=19530

MILVUS_USERNAME

MILVUS_PASSWORD

MILVUS_SECURE=

Weaviate vector db config

VECTOR_STORE_TYPE=Weaviate

WEAVIATE_URL=https://kt-region-m8hcy0wc.weaviate.network

***

                  WebServer Language Support                   

***

LANGUAGE=en

LANGUAGE=zh

***

**    PROXY_SERVER (openai interface | chatGPT proxy service), use chatGPT as your LLM.

** if your server can visit openai, please set PROXY_SERVER_URL=https://api.openai.com/v1/chat/completions

** else if you have a chatgpt proxy server, you can set PROXY_SERVER_URL={your-proxy-serverip:port/xxx}

***

PROXY_API_KEY={your-openai-sk} PROXY_SERVER_URL=https://api.openai.com/v1/chat/completions

from https://bard.google.com/     f12-> application-> __Secure-1PSID

BARD_PROXY_API_KEY={your-bard-token}

***

  PROXY_SERVER +                                             

***

Aliyun tongyi

TONGYI_PROXY_API_KEY={your-tongyi-sk}

Baidu wenxin

WEN_XIN_MODEL_VERSION={version}

WEN_XIN_API_KEY={your-wenxin-sk}

WEN_XIN_API_SECRET={your-wenxin-sct}

Zhipu

PROXY_SERVER_URL=https://open.bigmodel.cn/api/paas/v4 ZHIPU_PROXY_API_KEY=0e843c5ff6598a4d3bcdc3baddc80a4b.qNoZbSysGuNe1L8j

Baichuan

BAICHUN_MODEL_NAME={version}

BAICHUAN_PROXY_API_KEY={your-baichuan-sk}

BAICHUAN_PROXY_API_SECRET={your-baichuan-sct}

Xunfei Spark

XUNFEI_SPARK_API_VERSION={version}

XUNFEI_SPARK_APPID={your_app_id}

XUNFEI_SPARK_API_KEY={your_api_key}

XUNFEI_SPARK_API_SECRET={your_api_secret}

***

    SUMMARY_CONFIG                                             

***

SUMMARY_CONFIG=FAST

***

    MUlti-GPU                                                 

***

See https://developer.nvidia.com/blog/cuda-pro-tip-control-gpu-visibility-cuda_visible_devices/

If CUDA_VISIBLE_DEVICES is not configured, all available gpus will be used

CUDA_VISIBLE_DEVICES=0

You can configure the maximum memory used by each GPU.

MAX_GPU_MEMORY=16Gib

***

                         LOG                                   

***

FATAL, ERROR, WARNING, WARNING, INFO, DEBUG, NOTSET

DBGPT_LOG_LEVEL=INFO

LOG dir, default: ./logs

DBGPT_LOG_DIR=

-_- @.***

 

其中，我也设置了语言LANGUAGE=zh，但是也没有生效。我运行起来之后，还是英文。十分期待您的帮助和解答。祝您生活愉快。 ------------------ 原始邮件 ------------------ 发件人: "eosphoros-ai/DB-GPT" @.>; 发送时间: 2024年1月20日(星期六) 上午10:18 @.>; @.**@.>; 主题: Re: [eosphoros-ai/DB-GPT] [1211][模型不存在，请检查模型代码。] (error_code: 1) (Issue #1093)

can you show your .env setting about llm ?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

TaoPeiLing commented 9 months ago

你能展示一下你的 llm 的 .env 设置吗？

The issue has been resolved, but I still appreciate your reply very much.

alexlovy commented 7 months ago

how you resolved plz?

你能展示一下你的 llm 的 .env 设置吗？

The issue has been resolved, but I still appreciate your reply very much.