-
### Checklist
- [ ] 1. I have searched related issues but cannot get the expected help.
- [ ] 2. The bug has not been fixed in the latest version.
### Describe the bug
baichuan2 7b-13b cannot be 8-…
-
I found that in the benchmark/suite has the output time to first token. However, when I run `python benchmark.py --model meta-llama/Llama-2-7b-hf static --isl 128 --osl 128 --batch 1` an error occurs:…
-
### Summary
- Provide k-quant models
- Maintain existing gguf models
- Embedding models
- [x] [second-state/Nomic-embed-text-v1.5-Embedding-GGUF](https://huggingface.co/second-state/Nomic-…
-
I am trying to convert baichuan2-megatron to hf. When reading the code, i can not understand this part
```
def permute(x):
if revert:
return x.view(head_dim//2, 2, dim).transpo…
drxmy updated
10 months ago
-
评估使用的代码：https://github.com/baichuan-inc/Baichuan-7B/blob/main/evaluation/evaluate_mmlu.py
用bf16精度测试 llama2-13-hf 和 baichuan2-13b-base
llama2-13-hf: 0.550
baichuan2-13b-base: 0.564
改了一行代码，用fp32…
-
一张A100 40g显卡，服务器内存30g，本地下载了最新的baichuan2-7b-base模型文件，安装环境也是按照requirement的要求安装的，使用github提供的脚本在运行：
model = AutoModelForCausalLM.from_pretrained("/home/baichuan/baichuan_funtuning/Baichuan2/baichuan2_7b_…
-
执行python cli_demo.py， 会弹出对话框，但是输入首个问题，就会报错，提示如下：
Traceback (most recent call last):
File "/home/ubuntu/anaconda3/envs/baichuan2/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
se…
-
使用Baichuan2-7B-base作为预训练模型，belle-10k数据集进行微调，起始loss是2.x；
使用Baichuan2-13B-Chat作为预训练模型，belle-10k数据集进行微调，起始loss是2.x；
使用Baichuan2-13B-Base作为预训练模型，belle-10k数据集进行微调，起始loss是**6.x**；
这个正常吗？
-
在deepspeed启动训练的.py文件中使用AutoModelForCausalLM.from_pretrained加载baichuan-inc/Baichuan2-7B-Intermediate-Checkpoints中的checkpoint太慢，加载7B的模型都需要几十分钟。
但是在jupyter notebook中加载就很快，几十秒能完成。
或者用deepspeed加载7B-bas…
-
