./run.sh -c local -i 0 -b hf -m MiniChat-2-3B -t minichat报错

xiaoma1990-git commented 7 months ago

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

[X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

[X] 我已经搜索过FAQ | I have searched FAQ

当前行为 | Current Behavior

No response

期望行为 | Expected Behavior

No response

运行环境 | Environment

- OS:
- NVIDIA Driver:
- CUDA:
- docker:
- docker-compose:
- NVIDIA GPU:
- NVIDIA GPU Memory:

QAnything日志 | QAnything logs

qanything-container-local | �� LLM �� /workspace/qanything_local/logs/debug_logs/fastchat_logs/fschat_model_worker_7801.log ��Error... qanything-container-local | 2024-04-07 15:54:44 | ERROR | stderr | tokenizer.Load(self.vocab_file) qanything-container-local | 2024-04-07 15:54:44 | ERROR | stderr | File "/usr/local/lib/python3.10/dist-packages/sentencepiece/init.py", line 905, in Load qanything-container-local | 2024-04-07 15:54:44 | ERROR | stderr | return self.LoadFromFile(model_file) qanything-container-local | 2024-04-07 15:54:44 | ERROR | stderr | File "/usr/local/lib/python3.10/dist-packages/sentencepiece/init.py", line 310, in LoadFromFile qanything-container-local | 2024-04-07 15:54:44 | ERROR | stderr | return _sentencepiece.SentencePieceProcessor_LoadFromFile(self, arg) qanything-container-local | 2024-04-07 15:54:44 | ERROR | stderr | RuntimeError: Internal: src/sentencepiece_processor.cc(1101) [model_proto->ParseFromArray(serialized.data(), serialized.size())]

复现方法 | Steps To Reproduce

No response

备注 | Anything else?

No response

fivegg commented 7 months ago

我昨天使用MiniChat-2-3B也遇到了，48G内存，3060显卡，win11 wsl2。今天换成Qwen-1_8B-Chat，LLM启动成功了，正在测试中

xiaoma1990-git commented 7 months ago

NVIDIA GeForce RTX 2080 Ti 我的显卡是这个另外还有T4的好像只能弄1.8B 或者3B的模型硬件限制。。

fivegg commented 7 months ago

Qwen-1_8B-Chat可以运行，但是对话有问题， bash ./run.sh -c local -i 0 -b hf -m Qwen-1_8B-Chat -t qwen-7b-qanything。

vc5409ftu commented 3 months ago

3B要求显存得有10G以上

GOOD-N-LCM commented 3 months ago

我 2080ti 22G 没跑起来 3B模型............

GOOD-N-LCM commented 3 months ago

Qwen-1_8B-Chat

48g显存运行3B的那个模型也没起来么？我是2080ti 22g的启动失败了

netease-youdao / QAnything