-
在deepspeed启动训练的.py文件中使用AutoModelForCausalLM.from_pretrained加载baichuan-inc/Baichuan2-7B-Intermediate-Checkpoints中的checkpoint太慢，加载7B的模型都需要几十分钟。
但是在jupyter notebook中加载就很快，几十秒能完成。
或者用deepspeed加载7B-bas…
-
经过我的实验，当我设置input为
```
{'input_ids': tensor([[43707, 4007, 1833, 10329, 1836, 7293, 3799, 65, 22792, 2169,
4007, 1833, 2218, 2079, 65, 92413, 11721, 7293, 2835, 1754,
…
-
调用的是baichuan2flm.py转的是Baichuan2-7B-Chat模型时报错。
/fastllm/build/tools/baichuan2flm.py", line 12, in
model.to("cpu")
python3.10/site-packages/accelerate/big_modeling.py", line 415, in wrapper
…
-
when I run Baichuan2-7B-base model ,meet the issue that "position_ids" is not defined. And I checked the code and did not find the declaration of position_ids in the code
```
pt-p26l37d2-worker-0 …
-
when i use local model, it will happen
ValueError: Tokenizer class BaichuanTokenizer does not exist or is not currently imported.
so，what should i do
-
model_path = "/home/test/models/LLM/baichuan2-7b/pytorch"
# Load and optimize the INT4 model with IPEX
low_bit = "sym_int4"
model_int4 = BigdlForCausalLM.from_pretrained(model_path, load_in_low_b…
-
用的模型是Baichuan2-7B-Chat-4bits，用lora微调，双3090，不用lora会oom
训练数据是官方给的belle_chat_ramdon_10k.json
-
**例行检查**
[//]: # (方框内删除已有的空格，填 x 号)
+ [x] 我已确认目前没有类似 issue
+ [x] 我已确认我已升级到最新版本
+ [x] 我已完整查看过项目 README，尤其是常见问题部分
+ [x] 我理解并愿意跟进此 issue，协助测试和提供反馈
+ [x] 我理解并认可上述内容，并理解项目维护者精力有限，**不遵循规则的 issue 可能…
-
I am trying to deploy a Baichuan2-7B model on a machine with 2 Tesla V100 GPUs. Unfortunately each V100 has only 16GB memory.
I have applied INT8 weight-only quantization, so the size of the engine I…
-
刚下载的项目
使用官方提供的脚本
hostfile=""
deepspeed --hostfile=$hostfile fine-tune.py \
--report_to "none" \
--data_path "data/belle_chat_ramdon_10k.json" \
--model_name_or_path "baichuan-inc/Ba…