jiahe7ay MINI_LLM issues

jiahe7ay / MINI_LLM

This is a repository used by individuals to experiment and reproduce the pre-training process of LLM.

315 stars 50 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

采用4090多卡预训练的时候，出现权重文件保存错误

#30 2279072142 opened 1 month ago
2
数据截断问题

#29 zhangrui17 opened 3 months ago
2
数据预处理时的问题

#28 zhangrui17 opened 3 months ago
2
requirements.txt安装报错

#27 jzzzf opened 3 months ago
1
sft阶段不同卡显存占用不同

#26 lainxx opened 3 months ago
0
添加DPO对比链接

#25 wtxfrancise closed 3 months ago
0
添加DPO 相关代码

#24 wtxfrancise closed 3 months ago
0
预训练到特定step出现OutOfMemoryError

#23 Tshirt-hfk opened 4 months ago
4
预训wiki练数据处理问题

#22 daneren closed 4 months ago
1
24G4090 预训练OOM

#21 Galloroc1 closed 4 months ago
2
预训练初始模型

#20 xiaolling opened 4 months ago
3
预训练数据的细节问题

#19 awdrgyjilplij opened 4 months ago
3
add multi nodes multi gpus config

#18 lumiere-ml closed 4 months ago
0
在V100上训练总是出现loss为0，grad_norm为Nan的情况，不知是否有相似情况的？

#17 Willamjie opened 4 months ago
5
大哥我尝试将这个代码缩小到120m左右模型，为啥训练的时候进行对话回复，，，，这是什么情况

#16 kingpingyue opened 5 months ago
7
数据预处理问题

#15 wtxfrancise closed 4 months ago
1
reorg the folder

#14 ElegantLin closed 5 months ago
0
预训练数据问题

#13 lumiere-ml opened 5 months ago
2
Keesh0410 update readme sft link

#12 keesh0410 closed 5 months ago
0
修改了ReadMe的data_process.py 那个写错的代码.

#11 keesh0410 closed 5 months ago
0
优化 gen_sky 函数以处理整个文件夹的 JSONL 文件，并增强项目文档说明Update generate_data.py

#10 keesh0410 closed 5 months ago
0
预训练时遇到错误

#9 hulonghua-devin opened 5 months ago
6
遇到NaN or Inf found in input tensor.错误

#8 andyweiqiu opened 5 months ago
2
pretrain阶段的验证集数据是如何选取的呢？

#7 moyans closed 5 months ago
1
大佬，qwen最接近的是1.8B,咋训出来1.4B的？

#6 daizehua1 opened 5 months ago
4
大哥麻烦给个数据样本参考一下

#5 kingpingyue opened 5 months ago
6
请教在3090 * 8 的机器下，报错cuda out of memory怎么处理

#4 weiiWill closed 5 months ago
1
在安装flash-attn这个包的时候卡了特别特别久

#3 Zengfanxu1111 opened 5 months ago
2
请问这个工程能在8卡4090上复现吗？

#2 LYMDLUT opened 5 months ago
2
没有找到data_process.py文件

#1 Yaoisss closed 5 months ago
2