issues
search
jiahe7ay
/
MINI_LLM
This is a repository used by individuals to experiment and reproduce the pre-training process of LLM.
315
stars
50
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
采用4090多卡预训练的时候,出现权重文件保存错误
#30
2279072142
opened
1 month ago
2
数据截断问题
#29
zhangrui17
opened
3 months ago
2
数据预处理时的问题
#28
zhangrui17
opened
3 months ago
2
requirements.txt安装报错
#27
jzzzf
opened
3 months ago
1
sft阶段不同卡显存占用不同
#26
lainxx
opened
3 months ago
0
添加DPO对比链接
#25
wtxfrancise
closed
3 months ago
0
添加DPO 相关代码
#24
wtxfrancise
closed
3 months ago
0
预训练到特定step出现OutOfMemoryError
#23
Tshirt-hfk
opened
4 months ago
4
预训wiki练数据处理问题
#22
daneren
closed
4 months ago
1
24G4090 预训练OOM
#21
Galloroc1
closed
4 months ago
2
预训练初始模型
#20
xiaolling
opened
4 months ago
3
预训练数据的细节问题
#19
awdrgyjilplij
opened
4 months ago
3
add multi nodes multi gpus config
#18
lumiere-ml
closed
4 months ago
0
在V100上训练总是出现loss为0,grad_norm为Nan的情况,不知是否有相似情况的?
#17
Willamjie
opened
4 months ago
5
大哥 我尝试将这个代码缩小到120m左右模型,为啥训练的时候进行对话回复,,,,这是什么情况
#16
kingpingyue
opened
5 months ago
7
数据预处理问题
#15
wtxfrancise
closed
4 months ago
1
reorg the folder
#14
ElegantLin
closed
5 months ago
0
预训练数据问题
#13
lumiere-ml
opened
5 months ago
2
Keesh0410 update readme sft link
#12
keesh0410
closed
5 months ago
0
修改了ReadMe的data_process.py 那个写错的代码.
#11
keesh0410
closed
5 months ago
0
优化 gen_sky 函数以处理整个文件夹的 JSONL 文件,并增强项目文档说明Update generate_data.py
#10
keesh0410
closed
5 months ago
0
预训练时遇到错误
#9
hulonghua-devin
opened
5 months ago
6
遇到NaN or Inf found in input tensor.错误
#8
andyweiqiu
opened
5 months ago
2
pretrain阶段的验证集数据是如何选取的呢?
#7
moyans
closed
5 months ago
1
大佬,qwen最接近的是1.8B,咋训出来1.4B的?
#6
daizehua1
opened
5 months ago
4
大哥 麻烦给个数据样本参考一下
#5
kingpingyue
opened
5 months ago
6
请教在3090 * 8 的机器下,报错cuda out of memory怎么处理
#4
weiiWill
closed
5 months ago
1
在安装flash-attn这个包的时候卡了特别特别久
#3
Zengfanxu1111
opened
5 months ago
2
请问这个工程能在8卡4090上复现吗?
#2
LYMDLUT
opened
5 months ago
2
没有找到data_process.py文件
#1
Yaoisss
closed
5 months ago
2