dosubot[bot]
commented
6 months ago Hey there, @302658980! Long time no chat. Hope you're doing awesomely! 🚀
To replace or speed up your current "bge-large-zh" Embedding model located at "/root/model/bge-large-zh" in the MODEL_PATH dictionary, you can either update the path to a new model or choose a different model that might be faster or more efficient. Here's how you can do it based on the provided configuration file:
-
Replace with a New Model:
- If you have a new model that you believe is faster or more suitable, you can simply change the path of the
"bge-large-zh"key in theMODEL_PATHdictionary to point to the new model's location. For example, if the new model is located at/new/model/path, update the dictionary as follows:"bge-large-zh": "/new/model/path", - Ensure the new model is compatible with the rest of your setup.
- If you have a new model that you believe is faster or more suitable, you can simply change the path of the
-
Choose a Different Pre-configured Model:
- If you want to use a different pre-configured model that is expected to be faster, you can change the
EMBEDDING_MODELvariable to match the key of the desired model in theMODEL_PATHdictionary. For example, if you want to switch to"bge-base-zh-v1.5"which might be lighter and faster, update theEMBEDDING_MODELas follows:EMBEDDING_MODEL = "bge-base-zh-v1.5" - This change assumes that
"bge-base-zh-v1.5"is a suitable replacement for your needs. You might need to consider the trade-offs in model performance and speed.
- If you want to use a different pre-configured model that is expected to be faster, you can change the
-
Configuration Adjustments:
MODEL_ROOT_PATH: If your new model is located in a directory that is a child of a common root directory, you can setMODEL_ROOT_PATHto this common root directory. The program will automatically detect and load the model if its directory name matches theEMBEDDING_MODELor the keys/values inMODEL_PATH.EMBEDDING_DEVICE: Ensure this is set to the optimal device for running your model. For example, if you have a GPU available, setting it to"cuda"might speed up the model's performance.
Remember, when making changes, ensure that the new model is compatible with your application's requirements and that you have tested the model's performance to ensure it meets your needs.
The compatible models that can be used as a replacement for the "bge-large-zh" Embedding model in the Langchain-Chatchat codebase, as per the provided context, include:
- "bge-small-zh": "BAAI/bge-small-zh"
- "bge-base-zh": "BAAI/bge-base-zh"
- "bge-large-zh-noinstruct": "BAAI/bge-large-zh-noinstruct"
- "bge-base-zh-v1.5": "BAAI/bge-base-zh-v1.5"
- "bge-large-zh-v1.5": "BAAI/bge-large-zh-v1.5"
- "bge-m3": "BAAI/bge-m3"
These models are listed under the "embed_model" section in the provided configuration file, indicating they are intended for embedding purposes similar to "bge-large-zh".
Unfortunately, I wasn't able to find specific performance differences between the "bge-large-zh" Embedding model and the other compatible models listed in the Langchain-Chatchat codebase within the repository. You might need to conduct some performance tests to determine which model best suits your needs.
For more details, you can refer to the model configuration example in the repository.
302658980
github-actions[bot]
import os
可以指定一个绝对路径，统一存放所有的Embedding和LLM模型。
每个模型可以是一个单独的目录，也可以是某个目录下的二级子目录。
如果模型目录名称和 MODEL_PATH 中的 key 或 value 相同，程序会自动检测加载，无需修改 MODEL_PATH 中的路径。
MODEL_ROOT_PATH = ""
选用的 Embedding 名称
EMBEDDING_MODEL = "bge-large-zh"
Embedding 模型运行设备。设为"auto"会自动检测，也可手动设定为"cuda","mps","cpu"其中之一。
EMBEDDING_DEVICE = "auto"
如果需要在 EMBEDDING_MODEL 中增加自定义的关键字时配置
EMBEDDING_KEYWORD_FILE = "keywords.txt" EMBEDDING_MODEL_OUTPUT_PATH = "output"
要运行的 LLM 名称，可以包括本地模型和在线模型。列表中本地模型将在启动项目时全部加载。
列表中第一个模型将作为 API 和 WEBUI 的默认模型。
在这里，我们使用目前主流的两个离线模型，其中，chatglm3-6b 为默认加载模型。
如果你的显存不足，可使用 Qwen-1_8B-Chat, 该模型 FP16 仅需 3.8G显存。
LLM_MODELS = ["qwen-14b", "yi-34b"] # "Qwen-1_8B-Chat",
AgentLM模型的名称 (可以不指定，指定之后就锁定进入Agent之后的Chain的模型，不指定就是LLM_MODELS[0])
Agent_MODEL = None
LLM 运行设备。设为"auto"会自动检测，也可手动设定为"cuda","mps","cpu"其中之一。
LLM_DEVICE = "auto"
历史对话轮数
HISTORY_LEN = 3
大模型最长支持的长度，如果不填写，则使用模型默认的最大长度，如果填写，则为用户设定的最大长度
MAX_TOKENS = None
LLM通用对话参数
TEMPERATURE = 0.7
TOP_P = 0.95 # ChatOpenAI暂不支持该参数
ONLINE_LLM_MODEL = {
线上模型。请在server_config中为每个在线API设置不同的端口
}
在以下字典中修改属性值，以指定本地embedding模型存储位置。支持3种设置方法：
1、将对应的值修改为模型绝对路径
2、不修改此处的值（以 text2vec 为例）：
2.1 如果{MODEL_ROOT_PATH}下存在如下任一子目录：
- text2vec
- GanymedeNil/text2vec-large-chinese
- text2vec-large-chinese
2.2 如果以上本地路径不存在，则使用huggingface模型
MODEL_PATH = { "embed_model": { "ernie-tiny": "nghuyong/ernie-3.0-nano-zh", "ernie-base": "nghuyong/ernie-3.0-base-zh", "text2vec-base": "shibing624/text2vec-base-chinese", "text2vec": "GanymedeNil/text2vec-large-chinese", "text2vec-paraphrase": "shibing624/text2vec-base-chinese-paraphrase", "text2vec-sentence": "shibing624/text2vec-base-chinese-sentence", "text2vec-multilingual": "shibing624/text2vec-base-multilingual", "text2vec-bge-large-chinese": "shibing624/text2vec-bge-large-chinese", "m3e-small": "moka-ai/m3e-small", "m3e-base": "moka-ai/m3e-base", "m3e-large": "moka-ai/m3e-large", "bge-small-zh": "BAAI/bge-small-zh", "bge-base-zh": "BAAI/bge-base-zh", "bge-large-zh": "/root/model/bge-large-zh", "bge-large-zh-noinstruct": "BAAI/bge-large-zh-noinstruct", "bge-base-zh-v1.5": "BAAI/bge-base-zh-v1.5", "bge-large-zh-v1.5": "BAAI/bge-large-zh-v1.5", "piccolo-base-zh": "sensenova/piccolo-base-zh", "piccolo-large-zh": "sensenova/piccolo-large-zh", "nlp_gte_sentence-embedding_chinese-large": "damo/nlp_gte_sentence-embedding_chinese-large", "text-embedding-ada-002": "your OPENAI_API_KEY", },
}
通常情况下不需要更改以下内容
nltk 模型存储路径
NLTK_DATA_PATH = os.path.join(os.path.dirname(os.path.dirname(file)), "nltk_data")
VLLM_MODEL_DICT = { "aquila-7b": "BAAI/Aquila-7B", "aquilachat-7b": "BAAI/AquilaChat-7B",
}
你认为支持Agent能力的模型，可以在这里添加，添加后不会出现可视化界面的警告
经过我们测试，原生支持Agent的模型仅有以下几个
SUPPORT_AGENT_MODEL = [ "azure-api", "openai-api", "qwen-api", "Qwen", "chatglm3", "xinghuo-api", ] 要修改上面配置文件的哪些内容