search

QwenLM / Qwen2

Qwen2 is the large language model series developed by Qwen team, Alibaba Cloud.
6k stars 336 forks source link

使用vllm部署Qwen2-57B-A14B-chat报错assert loaded_weight.shape[parallel_dim] == self.org_vocab_size。 #729

Open 1006076811 opened 6 days ago

1006076811 commented 6 days ago

设备信息：8*A800 80G

启动命令如下

nohup python -m vllm.entrypoints.openai.api_server \
    --served-model-name Qwen2-57B-A14B-Instruct \
    --model /media/user/data_one/nlp_model/Qwen2-57B-A14B-Instruct \
    --tensor-parallel-size 4 \
    --gpu-memory-utilization 0.6 \
    --chat-template ./vllm_template/template_chatml.jinja \
    --port 7000 \
> logs_dir/run_vllm_model.log 2>&1 &

报错信息如下

Traceback (most recent call last):
  File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
KeyError: '/psm_2d5c8b03'
Traceback (most recent call last):
  File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
KeyError: '/psm_2d5c8b03'
Traceback (most recent call last):
  File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
KeyError: '/psm_2d5c8b03'
(VllmWorkerProcess pid=3477401) INFO 07-02 08:48:31 custom_all_reduce_utils.py:179] reading GPU P2P access cache from /root/.config/vllm/gpu_p2p_access_cache_for_0,1,2,3.json
(VllmWorkerProcess pid=3477399) INFO 07-02 08:48:31 custom_all_reduce_utils.py:179] reading GPU P2P access cache from /root/.config/vllm/gpu_p2p_access_cache_for_0,1,2,3.json
(VllmWorkerProcess pid=3477400) INFO 07-02 08:48:31 custom_all_reduce_utils.py:179] reading GPU P2P access cache from /root/.config/vllm/gpu_p2p_access_cache_for_0,1,2,3.json
INFO 07-02 08:48:31 custom_all_reduce_utils.py:179] reading GPU P2P access cache from /root/.config/vllm/gpu_p2p_access_cache_for_0,1,2,3.json
[rank0]: Traceback (most recent call last):
[rank0]:   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/runpy.py", line 196, in _run_module_as_main
[rank0]:     return _run_code(code, main_globals, None,
[rank0]:   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/runpy.py", line 86, in _run_code
[rank0]:     exec(code, run_globals)
[rank0]:   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 196, in <module>
[rank0]:     engine = AsyncLLMEngine.from_engine_args(
[rank0]:   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 398, in from_engine_args
[rank0]:     engine = cls(
[rank0]:   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 349, in __init__
[rank0]:     self.engine = self._init_engine(*args, **kwargs)
[rank0]:   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 473, in _init_engine
[rank0]:     return engine_class(*args, **kwargs)
[rank0]:   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 223, in __init__
[rank0]:     self.model_executor = executor_class(
[rank0]:   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/executor/multiproc_gpu_executor.py", line 142, in __init__
[rank0]:     super().__init__(*args, **kwargs)
[rank0]:   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/executor/distributed_gpu_executor.py", line 25, in __init__
[rank0]:     super().__init__(*args, **kwargs)
[rank0]:   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/executor/executor_base.py", line 41, in __init__
[rank0]:     self._init_executor()
[rank0]:   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/executor/multiproc_gpu_executor.py", line 66, in _init_executor
[rank0]:     self._run_workers("load_model",
[rank0]:   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/executor/multiproc_gpu_executor.py", line 119, in _run_workers
[rank0]:     driver_worker_output = driver_worker_method(*args, **kwargs)
[rank0]:   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/worker/worker.py", line 122, in load_model
[rank0]:     self.model_runner.load_model()
[rank0]:   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/worker/model_runner.py", line 148, in load_model
[rank0]:     self.model = get_model(
[rank0]:   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/model_executor/model_loader/__init__.py", line 21, in get_model
[rank0]:     return loader.load_model(model_config=model_config,
[rank0]:   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/model_executor/model_loader/loader.py", line 264, in load_model
[rank0]:     model.load_weights(
[rank0]:   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/model_executor/models/qwen2_moe.py", line 465, in load_weights
[rank0]:     weight_loader(param, loaded_weight)
[rank0]:   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/model_executor/layers/vocab_parallel_embedding.py", line 292, in weight_loader
[rank0]:     assert loaded_weight.shape[parallel_dim] == self.org_vocab_size
[rank0]: AssertionError
(VllmWorkerProcess pid=3477399) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226] Exception in worker VllmWorkerProcess while processing method load_model: , Traceback (most recent call last):
(VllmWorkerProcess pid=3477399) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/executor/multiproc_worker_utils.py", line 223, in _run_worker_process
(VllmWorkerProcess pid=3477399) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     output = executor(*args, **kwargs)
(VllmWorkerProcess pid=3477399) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/worker/worker.py", line 122, in load_model
(VllmWorkerProcess pid=3477399) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     self.model_runner.load_model()
(VllmWorkerProcess pid=3477399) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/worker/model_runner.py", line 148, in load_model
(VllmWorkerProcess pid=3477399) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     self.model = get_model(
(VllmWorkerProcess pid=3477399) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/model_executor/model_loader/__init__.py", line 21, in get_model
(VllmWorkerProcess pid=3477399) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     return loader.load_model(model_config=model_config,
(VllmWorkerProcess pid=3477399) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/model_executor/model_loader/loader.py", line 264, in load_model
(VllmWorkerProcess pid=3477399) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     model.load_weights(
(VllmWorkerProcess pid=3477399) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/model_executor/models/qwen2_moe.py", line 465, in load_weights
(VllmWorkerProcess pid=3477399) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     weight_loader(param, loaded_weight)
(VllmWorkerProcess pid=3477399) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/model_executor/layers/vocab_parallel_embedding.py", line 292, in weight_loader
(VllmWorkerProcess pid=3477399) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     assert loaded_weight.shape[parallel_dim] == self.org_vocab_size
(VllmWorkerProcess pid=3477399) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226] AssertionError
(VllmWorkerProcess pid=3477399) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226] 
(VllmWorkerProcess pid=3477400) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226] Exception in worker VllmWorkerProcess while processing method load_model: , Traceback (most recent call last):
(VllmWorkerProcess pid=3477400) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/executor/multiproc_worker_utils.py", line 223, in _run_worker_process
(VllmWorkerProcess pid=3477400) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     output = executor(*args, **kwargs)
(VllmWorkerProcess pid=3477400) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/worker/worker.py", line 122, in load_model
(VllmWorkerProcess pid=3477400) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     self.model_runner.load_model()
(VllmWorkerProcess pid=3477400) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/worker/model_runner.py", line 148, in load_model
(VllmWorkerProcess pid=3477400) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     self.model = get_model(
(VllmWorkerProcess pid=3477400) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/model_executor/model_loader/__init__.py", line 21, in get_model
(VllmWorkerProcess pid=3477400) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     return loader.load_model(model_config=model_config,
(VllmWorkerProcess pid=3477400) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/model_executor/model_loader/loader.py", line 264, in load_model
(VllmWorkerProcess pid=3477400) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     model.load_weights(
(VllmWorkerProcess pid=3477400) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/model_executor/models/qwen2_moe.py", line 465, in load_weights
(VllmWorkerProcess pid=3477400) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     weight_loader(param, loaded_weight)
(VllmWorkerProcess pid=3477400) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/model_executor/layers/vocab_parallel_embedding.py", line 292, in weight_loader
(VllmWorkerProcess pid=3477400) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     assert loaded_weight.shape[parallel_dim] == self.org_vocab_size
(VllmWorkerProcess pid=3477400) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226] AssertionError
(VllmWorkerProcess pid=3477400) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226] 
(VllmWorkerProcess pid=3477401) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226] Exception in worker VllmWorkerProcess while processing method load_model: , Traceback (most recent call last):
(VllmWorkerProcess pid=3477401) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/executor/multiproc_worker_utils.py", line 223, in _run_worker_process
(VllmWorkerProcess pid=3477401) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     output = executor(*args, **kwargs)
(VllmWorkerProcess pid=3477401) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/worker/worker.py", line 122, in load_model
(VllmWorkerProcess pid=3477401) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     self.model_runner.load_model()
(VllmWorkerProcess pid=3477401) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/worker/model_runner.py", line 148, in load_model
(VllmWorkerProcess pid=3477401) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     self.model = get_model(
(VllmWorkerProcess pid=3477401) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/model_executor/model_loader/__init__.py", line 21, in get_model
(VllmWorkerProcess pid=3477401) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     return loader.load_model(model_config=model_config,
(VllmWorkerProcess pid=3477401) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/model_executor/model_loader/loader.py", line 264, in load_model
(VllmWorkerProcess pid=3477401) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     model.load_weights(
(VllmWorkerProcess pid=3477401) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/model_executor/models/qwen2_moe.py", line 465, in load_weights
(VllmWorkerProcess pid=3477401) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     weight_loader(param, loaded_weight)
(VllmWorkerProcess pid=3477401) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]   File "/media/user/data_one/miniconda3/envs/qwen2/lib/python3.10/site-packages/vllm/model_executor/layers/vocab_parallel_embedding.py", line 292, in weight_loader
(VllmWorkerProcess pid=3477401) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226]     assert loaded_weight.shape[parallel_dim] == self.org_vocab_size
(VllmWorkerProcess pid=3477401) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226] AssertionError
(VllmWorkerProcess pid=3477401) ERROR 07-02 08:48:32 multiproc_worker_utils.py:226] 
INFO 07-02 08:48:35 multiproc_worker_utils.py:123] Killing local vLLM worker processes
[rank0]:[W CudaIPCTypes.cpp:16] Producer process has been terminated before all shared CUDA tensors released. See Note [Sharing CUDA tensors]

pip list如下

Package                           Version
--------------------------------- --------------
accelerate                        0.29.3
aiofiles                          23.2.1
aiohttp                           3.9.5
aiosignal                         1.3.1
altair                            5.3.0
annotated-types                   0.7.0
anyio                             4.4.0
argon2-cffi                       23.1.0
argon2-cffi-bindings              21.2.0
arrow                             1.3.0
asttokens                         2.4.1
async-lru                         2.0.4
async-timeout                     4.0.3
attrs                             23.2.0
Babel                             2.14.0
backcall                          0.2.0
beautifulsoup4                    4.12.3
bleach                            6.1.0
camelot-py                        0.11.0
certifi                           2024.6.2
cffi                              1.16.0
chardet                           5.2.0
charset-normalizer                3.3.2
click                             8.1.7
cloudpickle                       3.0.0
cmake                             3.29.6
comm                              0.2.2
confluent-kafka                   2.3.0
contourpy                         1.2.1
cryptography                      42.0.5
cycler                            0.12.1
dashscope                         1.17.0
dataclasses-json                  0.6.4
datasets                          2.20.0
debugpy                           1.8.1
decorator                         5.1.1
defusedxml                        0.7.1
dill                              0.3.8
diskcache                         5.6.3
distro                            1.9.0
dnspython                         2.6.1
docopt                            0.6.2
docx2txt                          0.8
ecdsa                             0.19.0
einops                            0.7.0
email_validator                   2.2.0
environs                          9.5.0
et-xmlfile                        1.1.0
exceptiongroup                    1.2.1
executing                         2.0.1
fastapi                           0.111.0
fastapi-cli                       0.0.4
fastjsonschema                    2.19.1
ffmpy                             0.3.2
filelock                          3.15.4
FlagEmbedding                     1.2.9
flash-attn                        2.5.7
fonttools                         4.51.0
fqdn                              1.5.1
frozenlist                        1.4.1
fsspec                            2024.5.0
gradio                            3.50.2
gradio_client                     0.6.1
greenlet                          3.0.3
grpcio                            1.58.0
h11                               0.14.0
html2text                         2024.2.26
httpcore                          1.0.5
httptools                         0.6.1
httpx                             0.27.0
huggingface-hub                   0.23.4
idna                              3.7
importlib_resources               6.4.0
iniconfig                         2.0.0
interegular                       0.3.3
ipykernel                         6.29.4
ipython                           8.12.3
ipywidgets                        8.1.2
isoduration                       20.11.0
jedi                              0.19.1
jieba                             0.42.1
Jinja2                            3.1.4
joblib                            1.4.0
json5                             0.9.25
jsonlines                         4.0.0
jsonpatch                         1.33
jsonpointer                       2.4
jsonschema                        4.22.0
jsonschema-specifications         2023.12.1
jupyter                           1.0.0
jupyter_client                    8.6.1
jupyter-console                   6.6.3
jupyter_core                      5.7.2
jupyter-events                    0.10.0
jupyter-lsp                       2.2.5
jupyter_server                    2.14.0
jupyter_server_terminals          0.5.3
jupyterlab                        4.1.6
jupyterlab_pygments               0.3.0
jupyterlab_server                 2.26.0
jupyterlab_widgets                3.0.10
kiwisolver                        1.4.5
langchain                         0.1.16
langchain-community               0.0.34
langchain-core                    0.1.45
langchain-text-splitters          0.0.1
langsmith                         0.1.49
lark                              1.1.9
llvmlite                          0.43.0
lm-format-enforcer                0.10.1
lxml                              5.2.1
markdown-it-py                    3.0.0
MarkupSafe                        2.1.5
marshmallow                       3.21.1
matplotlib                        3.8.4
matplotlib-inline                 0.1.7
mdurl                             0.1.2
minio                             7.2.5
mistune                           3.0.2
mpmath                            1.3.0
msgpack                           1.0.8
multidict                         6.0.5
multiprocess                      0.70.16
mypy-extensions                   1.0.0
nbclient                          0.10.0
nbconvert                         7.16.3
nbformat                          5.10.4
nest-asyncio                      1.6.0
networkx                          3.3
ninja                             1.11.1.1
notebook                          7.1.3
notebook_shim                     0.2.4
numba                             0.60.0
numpy                             1.26.4
nvidia-cublas-cu12                12.1.3.1
nvidia-cuda-cupti-cu12            12.1.105
nvidia-cuda-nvrtc-cu12            12.1.105
nvidia-cuda-runtime-cu12          12.1.105
nvidia-cudnn-cu12                 8.9.2.26
nvidia-cufft-cu12                 11.0.2.54
nvidia-curand-cu12                10.3.2.106
nvidia-cusolver-cu12              11.4.5.107
nvidia-cusparse-cu12              12.1.0.106
nvidia-ml-py                      12.555.43
nvidia-nccl-cu12                  2.20.5
nvidia-nvjitlink-cu12             12.5.82
nvidia-nvtx-cu12                  12.1.105
openai                            1.35.7
openpyxl                          3.1.2
orjson                            3.10.5
outlines                          0.0.46
overrides                         7.7.0
packaging                         24.1
pandas                            2.2.2
pandocfilters                     1.5.1
parso                             0.8.4
pdfminer.six                      20231228
pdfplumber                        0.11.0
pexpect                           4.9.0
pickleshare                       0.7.5
pillow                            10.4.0
pip                               23.3.1
pipreqs                           0.5.0
platformdirs                      4.2.0
pluggy                            1.5.0
prometheus_client                 0.20.0
prometheus-fastapi-instrumentator 7.0.0
prompt-toolkit                    3.0.43
protobuf                          5.27.2
psutil                            6.0.0
ptyprocess                        0.7.0
pure-eval                         0.2.2
py-cpuinfo                        9.0.0
pyairports                        2.1.1
pyarrow                           16.1.0
pyarrow-hotfix                    0.6
pyasn1                            0.6.0
pycountry                         24.6.1
pycparser                         2.22
pycryptodome                      3.20.0
pydantic                          2.8.0
pydantic_core                     2.20.0
pydub                             0.25.1
Pygments                          2.18.0
pymilvus                          2.3.3
PyMySQL                           1.1.0
pynvml                            11.5.0
pyparsing                         3.1.2
pypdf                             4.2.0
PyPDF2                            3.0.1
pypdfium2                         4.30.0
pytest                            8.1.1
python-dateutil                   2.9.0.post0
python-docx                       1.1.2
python-dotenv                     1.0.1
python-jose                       3.3.0
python-json-logger                2.0.7
python-multipart                  0.0.9
python-pptx                       0.6.23
pytz                              2024.1
PyYAML                            6.0.1
pyzmq                             26.0.0
qtconsole                         5.5.1
QtPy                              2.4.1
ray                               2.31.0
redis                             5.0.1
referencing                       0.35.1
regex                             2024.5.15
requests                          2.32.3
rfc3339-validator                 0.1.4
rfc3986-validator                 0.1.1
rich                              13.7.1
rpds-py                           0.18.1
rsa                               4.9
safetensors                       0.4.3
scikit-learn                      1.4.2
scipy                             1.13.0
seaborn                           0.13.2
semantic-version                  2.10.0
Send2Trash                        1.8.3
sentence-transformers             2.7.0
sentencepiece                     0.2.0
setuptools                        68.2.2
shellingham                       1.5.4
simpleflake                       0.1.5
six                               1.16.0
sniffio                           1.3.1
soupsieve                         2.5
SQLAlchemy                        2.0.22
stack-data                        0.6.3
starlette                         0.37.2
sympy                             1.12.1
tabula-py                         2.9.2
tabulate                          0.9.0
tenacity                          8.2.3
terminado                         0.18.1
threadpoolctl                     3.4.0
tiktoken                          0.7.0
tinycss2                          1.2.1
tokenizers                        0.19.1
tomli                             2.0.1
toolz                             0.12.1
torch                             2.3.0
tornado                           6.4
tqdm                              4.66.4
traitlets                         5.14.2
transformers                      4.42.3
triton                            2.3.0
typer                             0.12.3
types-python-dateutil             2.9.0.20240316
typing_extensions                 4.12.2
typing-inspect                    0.9.0
tzdata                            2024.1
ujson                             5.10.0
uri-template                      1.3.0
urllib3                           2.2.2
uvicorn                           0.30.1
uvloop                            0.19.0
vllm                              0.5.0.post1
vllm-flash-attn                   2.5.9
watchfiles                        0.22.0
wcwidth                           0.2.13
webcolors                         1.13
webencodings                      0.5.1
websocket-client                  1.7.0
websockets                        12.0
wheel                             0.41.2
widgetsnbextension                4.0.10
xformers                          0.0.26.post1
XlsxWriter                        3.2.0
xxhash                            3.4.1
yarg                              0.1.9
yarl                              1.9.4
MikeJackOne commented 6 days ago

8*A800?你在哪个公司？还招人不？

AU-CPU commented 4 hours ago

Have you solved the problem of inconsistent vocab_size? If you solved it, can you show me how you did it?