python startup.py -a后出现问题

cstkn commented 7 months ago

手动下载的模型不知道要放在哪里能被正常检测，并且出现以下报错 2024-03-01 18:48:26,514 - startup.py[line:655] - INFO: 正在启动服务： 2024-03-01 18:48:26,514 - startup.py[line:656] - INFO: 如需查看 llm_api 日志，请前往 D:\wujiaping\chatglm\langchain-chatchat\logs D:\wujiaping\chatglm.venv\Lib\site-packages\langchain_core_api\deprecation.py:117: LangChainDeprecationWarning: 模型启动功能将于 Langchain-Chatchat 0.3.x重写,支持更多模式和加速启动，0.2.x中相关功能将废弃 warn_deprecated( 2024-03-01 18:48:30 | INFO | model_worker | Register to controller 2024-03-01 18:48:30 | ERROR | stderr | INFO: Started server process [5192] 2024-03-01 18:48:30 | ERROR | stderr | INFO: Waiting for application startup. 2024-03-01 18:48:30 | ERROR | stderr | INFO: Application startup complete. 2024-03-01 18:48:30 | ERROR | stderr | INFO: Uvicorn running on http://127.0.0.1:20000 (Press CTRL+C to quit) 2024-03-01 18:48:31 | INFO | model_worker | Loading the model ['chatglm3-6b'] on worker 9f519a57 ... 2024-03-01 18:48:38 | INFO | stdout | INFO: 127.0.0.1:53553 - "GET / HTTP/1.1" 404 Not Found

dosubot[bot] commented 7 months ago

Hey there, @cstkn! Good to see you back. Running into some new challenges, I see? Let’s dive into it.

Based on the issues you've encountered, it seems like the application is having trouble locating the model files in your local environment and is attempting to download them from the internet, which is causing the connection timeout errors. Here are some steps you can take to resolve these issues:

Modify the tokenizer_config.json file: As suggested in the issue #3074, you can modify the tokenizer_config.json file for the chatglm3-6b model. Change the auto_map entry from:

  "auto_map": {
    "AutoTokenizer": [
      "THUDM/chatglm3-6b--tokenization_chatglm.ChatGLMTokenizer",
      null
    ]
  },

to:

  "auto_map": {
    "AutoTokenizer": [
      "tokenization_chatglm.ChatGLMTokenizer",
      null
    ]
  },

This change should be made in the tokenizer_config.json file located in the directory of the chatglm3-6b model.

Set the MODEL_ROOT_PATH: Ensure that MODEL_ROOT_PATH in your configuration file points to the absolute path where all your models, including chatglm3-6b and bge-large-zh-v1.5, are stored. If this path is not set or incorrect, the application cannot locate your offline models.
Verify Model Directory Structure: The models should be in directories that match their keys in the MODEL_PATH dictionary or their repo_id as specified. For chatglm3-6b, the directory should be named THUDM/chatglm3-6b or simply chatglm3-6b if placed directly under MODEL_ROOT_PATH. Similarly, for bge-large-zh-v1.5, the directory should be named BAAI/bge-large-zh-v1.5 or simply bge-large-zh-v1.5.
Update MODEL_PATH if Necessary: If your models are stored in a different structure than the default expected paths, update the MODEL_PATH dictionary in your configuration file to reflect the correct paths.
Rename or Copy the Configuration File: If you haven't already, make sure to rename configs/model_config.py.example to configs/model_config.py (or the appropriate name expected by your application) and ensure your changes are saved in this file.
Check Permissions: Ensure that the user running python3 startup.py -a has read access to the directories where the models are stored.

If you're still encountering issues, please provide more detailed information about your system setup and the exact error messages you're seeing. This will help in diagnosing the problem and providing a more specific solution.

Sources

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.

cstkn commented 7 months ago

https://huggingface.co/BAAI/bge-large-zh 该文件应该放在哪里或者在哪里指定@dosu-bot

dosubot[bot] commented 7 months ago

Based on the context provided, it seems like you're having trouble with the placement of the model file downloaded from https://huggingface.co/BAAI/bge-large-zh. To ensure the model file is properly detected by your application and to avoid LocalEntryNotFoundError, OSError, and connection timeout errors when connecting to 'https://huggingface.co', you should place the model file in a directory under the path specified by MODEL_ROOT_PATH in your configs/model_config.py.example file.

The directory should be named either bge-large-zh or BAAI/bge-large-zh to match the key or value specified in the MODEL_PATH dictionary for the bge-large-zh model. If MODEL_ROOT_PATH is left as an empty string, you should define an absolute path for it where you prefer to store your models.

For example, if you decide to store your models in /path/to/models, you should:

Set MODEL_ROOT_PATH = "/path/to/models" in your model_config.py.example file.
Create a directory named bge-large-zh or BAAI/bge-large-zh under /path/to/models.
Place the downloaded model file inside this directory.

This setup will allow your application to automatically detect and load the bge-large-zh model without needing to modify the MODEL_PATH dictionary or encountering errors due to missing models or failed connections to Hugging Face's servers.

You can refer to the following issues in the Langchain-Chatchat repository that have been resolved and are related to your problem:

Please note that the default value of MODEL_ROOT_PATH in the configs/model_config.py.example file in the Langchain-Chatchat codebase is an empty string (""). You can find this information in the configs/model_config.py.example file.

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Sources

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.

cstkn commented 7 months ago

@dosu-bot 我手动下载了https://huggingface.co/BAAI/bge-large-zh 中的文件，而不是用git，我应该将它们放在哪里，或在那个文件中修改路径，能保证这个baai文件能被正常识别

dosubot[bot] commented 7 months ago

Based on the context provided, here are the specific steps you can follow to manually download the model files from 'https://huggingface.co/BAAI/bge-large-zh' and correctly set the MODEL_ROOT_PATH in the configs/model_config.py.example file:

Download the Model Files:
- Visit 'https://huggingface.co/BAAI/bge-large-zh'.
- Download the model files manually to your local machine.
Choose a Location for Model Files:
- Decide on a directory where you want to store the model files. This directory will be referred to as <MODEL_DIRECTORY> in the following steps. For example, it could be something like /home/user/models/bge-large-zh.
Place the Downloaded Files:
- Place the downloaded model files into the <MODEL_DIRECTORY>.
Modify the MODEL_ROOT_PATH in configs/model_config.py.example:
- Open the file configs/model_config.py.example in a text editor.
- Locate the line MODEL_ROOT_PATH = "".
- Modify it to point to the parent directory of <MODEL_DIRECTORY>. If <MODEL_DIRECTORY> is /home/user/models/bge-large-zh, and /home/user/models is the directory where you plan to store all your models, then you should set MODEL_ROOT_PATH = "/home/user/models".
```
MODEL_ROOT_PATH = "/home/user/models"
```
Ensure Correct Model Path Configuration:
- Given the context provided, the model configuration uses keys to map to specific model paths in the MODEL_PATH dictionary. Since you are using the model bge-large-zh, which is already specified in the MODEL_PATH dictionary under the embed_model key as "bge-large-zh": "BAAI/bge-large-zh", the program should automatically detect and load the model from the <MODEL_DIRECTORY> under MODEL_ROOT_PATH without further modifications, assuming the directory name matches the key or value specified in MODEL_PATH.

By following these steps, you should be able to avoid LocalEntryNotFoundError, OSError, and connection timeout errors related to loading the model locally, as the program will now use the local copy of the model files instead of trying to download them from 'https://huggingface.co'.

As for the structure of the MODEL_PATH dictionary in the configs/model_config.py.example file, it is structured into two main keys: embed_model and llm_model. Each of these keys maps to a dictionary where the keys are model identifiers (e.g., "ernie-tiny", "chatglm2-6b") and the values are either model paths or model names (e.g., "nghuyong/ernie-3.0-nano-zh", "THUDM/chatglm2-6b"). This structure allows for specifying the storage locations of local embedding models and LLM models, with support for both absolute paths and model names from repositories like Hugging Face.

MODEL_PATH = {
    "embed_model": {
        "ernie-tiny": "nghuyong/ernie-3.0-nano-zh",
        "ernie-base": "nghuyong/ernie-3.0-base-zh",
        ...
        "text-embedding-ada-002": "your OPENAI_API_KEY",
    },
    "llm_model": {
        "chatglm2-6b": "THUDM/chatglm2-6b",
        "chatglm2-6b-32k": "THUDM/chatglm2-6b-32k",
        ...
        "opt-iml-max-30b": "facebook/opt-iml-max-30b",
    }
}

You can refer to the configs/model_config.py.example file for more details.

I hope this helps! If you have any further questions or run into any issues, feel free to ask.

Sources

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.

cstkn commented 7 months ago

CodeSlogan commented 7 months ago

放在哪里没有特别的要求，把路径改对就行，路径在config目录下的model_config.py文件

cstkn commented 7 months ago

放在哪里没有特别的要求，把路径改对就行，路径在config目录下的model_config.py文件 bge-large-zh和chatglm3-6b都放到一个父目录里吗

zRzRzRzRzRzRzR commented 7 months ago

ssertionError: Torch not compiled with CUDA enabled 你环境都不对啊，检查环境吧

wangqinbo0609 commented 6 months ago

2024-03-07 21:02:10,494 - startup.py[line:655] - INFO: 正在启动服务： 2024-03-07 21:02:10,494 - startup.py[line:656] - INFO: 如需查看 llm_api 日志，请前往 D:\code\ai\windows_download\Langchain-Chatchat\logs C:\Users\wangqinbo\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain_core_api\deprecation.py:117: LangChainDeprecationWarning: 模型启动功能将于 Langchain-Chatchat 0.3.x重写,支持更多模式和加速启动，0.2.x中相关功能将废弃 warn_deprecated( 2024-03-07 21:02:26 | INFO | model_worker | Register to controller 2024-03-07 21:02:26 | ERROR | stderr | INFO: Started server process [2696] 2024-03-07 21:02:26 | ERROR | stderr | INFO: Waiting for application startup. 2024-03-07 21:02:26 | ERROR | stderr | INFO: Application startup complete. 2024-03-07 21:02:26 | ERROR | stderr | INFO: Uvicorn running on http://127.0.0.1:20000 (Press CTRL+C to quit) 2024-03-07 21:02:27 | INFO | model_worker | Loading the model ['chatglm3-6b'] on worker 8cc26e63 ... 2024-03-07 21:02:28 | WARNING | transformers_modules.chatglm3-6b.tokenization_chatglm | Setting eos_token is not supported, use the default one. 2024-03-07 21:02:28 | WARNING | transformers_modules.chatglm3-6b.tokenization_chatglm | Setting pad_token is not supported, use the default one. 2024-03-07 21:02:28 | WARNING | transformers_modules.chatglm3-6b.tokenization_chatglm | Setting unk_token is not supported, use the default one.

transformers_modules.chatglm3-6b.tokenization_chatglm | Setting unk_token is not supported, use the default one.

chatchat-space / Langchain-Chatchat