issues
search
OpenLMLab
/
LOMO
LOMO: LOw-Memory Optimization
MIT License
973
stars
68
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
One confusion about the LOMO paper
#79
Pairshoe
closed
4 months ago
1
FIX: avoid math errors for edge cases
#78
younesbelkada
closed
4 months ago
2
MOE and Custom Finetuning
#77
DRXD1000
closed
5 months ago
0
fix linked images in PyPI page
#76
Borda
closed
6 months ago
1
package lomo_optim
#75
KYLN24
closed
6 months ago
1
installable package
#74
Borda
closed
6 months ago
2
请问adalomo可以支持用transformer中的trainer训练么？或者未来有可能实现么？
#73
lyt719
closed
2 months ago
20
Serious conclusion: LOMO does not significantly reduce GPU memory usage！
#72
misonsky
opened
8 months ago
17
How to load a 65B model on 24G GPU memory?
#71
misonsky
closed
8 months ago
37
实测LOMO++ deepspeed zero2 7b qlora llama 显存占用 感觉比正常的qlora + deepspeed zero2 显存占用大1倍
#70
zlh1992
opened
8 months ago
1
Can you provide detailed dependency versions?
#69
misonsky
opened
8 months ago
0
adalomo在使用chatglm2模型出现错误
#68
JorunoJobana
opened
10 months ago
2
adalomo 试图在 llama 2 70b 模型训练时出现 NCCL communicator 这类超时的错误
#67
HappyLynn
opened
10 months ago
5
Reproduce the results for LOMO
#66
shawnricecake
closed
10 months ago
2
eval environment for opencompass
#65
KaiLv69
opened
10 months ago
4
Instructions for evaluation datasets
#64
KaiLv69
closed
10 months ago
3
adalomo optimizer error
#63
shawnricecake
closed
10 months ago
15
Evaluation of Fine-tuned model with Adalomo
#62
sglucas
closed
9 months ago
1
cannot find adalomo class
#60
Yeojoon
closed
10 months ago
2
model merge error
#59
shawnricecake
closed
10 months ago
18
Mistral Support
#58
freegheist
closed
10 months ago
1
hook函数之所以再需要额外操作一次，那里的描述我觉得有些不妥，也或者我理解有问题
#57
LaosGAmin
closed
11 months ago
1
Runtime error on 2nd epoch, trying lora only
#56
wasifferoze
opened
11 months ago
0
a bug found in save_model of LOMOTrainer
#54
DingQiang2018
opened
1 year ago
10
LlaMA-7B + LoRA在16GB的V100上OOM
#53
zhenqincn
opened
1 year ago
2
Customized loss value
#52
ZN1010
opened
1 year ago
4
请问自定义Dataset只能是classification的数据集吗
#51
Tonystark64
opened
1 year ago
1
小学习率问题
#50
wpcool
opened
1 year ago
1
选取bloom-1b7作为model和wic作为数据集出现IndexError: tuple index out of range
#49
Tonystark64
opened
1 year ago
1
CLIP梯度和梯度overflow的影响
#48
tzjtatata
opened
1 year ago
2
为什么LOMO并没有火起来呢？
#47
Flywolfs
opened
1 year ago
5
LOMO是否支持bfloat16模型的训练？
#46
Wangyupei
closed
1 year ago
6
Memory Usage continues to grow
#45
Jetcodery
opened
1 year ago
5
Is LOMO a concurrent work of the official implementation?
#43
DesperateExplorer
closed
1 year ago
4
batch size开2后一直提示gradient overflow。。
#42
00drdelius
closed
1 year ago
16
关于代码理解和显存占用的问题
#41
anbyaa
opened
1 year ago
1
关于微调llama-65b的疑问
#40
Facico
closed
1 year ago
2
Functions to measure the memory usage
#38
JiaxiangRen
closed
1 year ago
1
公式4疑问
#37
yaorong1996
closed
1 year ago
1
How to calculate the used GPU memory for each part as in the paper?
#36
liming-ai
opened
1 year ago
2
LOMO+QLoRA简单更改后的报错
#35
00drdelius
closed
1 year ago
7
请教个问题，LLM 训练会存在 micro-batch 之间需要累积梯度的场景，这种场景也会有优化吗？
#34
nullnonenilNULL
closed
1 year ago
1
LORA+LOMO distributed learning
#33
JiaxiangRen
closed
1 year ago
2
type object 'torch._C._distributed_c10d.ReduceOp' has no attribute 'AVG'
#32
season1blue
closed
10 months ago
4
Key Error: LOCAL_RANK
#31
snykral
closed
1 year ago
1
about torch.stack(self.grad_norms)
#30
jinzitian
opened
1 year ago
3
我使用了Resnet50+LOMO优化器，使用cpu去跑，系统内存相比sgd 没有任何变化，请问合理吗
#29
yaocy
closed
1 year ago
0
llama-33B/llama-65B均报OOM，8*V100跑不起来怎么回事呢？
#28
alisyzhu
opened
1 year ago
7
Some confusion about the method of the paper
#27
JorunoJobana
opened
1 year ago
3
Memory consumption first grows up then falls down.
#26
zhenqin96
opened
1 year ago
3
Next