-
**Is your feature request related to a problem? Please describe.**
LAMB optimizer does not support BF16 training.
When I used the LAMB optimizer with BF16 training, I met the error
```
DeepSpeed …
-
-
### Describe the bug
**My launch_inpaint.sh**
export LD_LIBRARY_PATH=/u…
-
Dear @salman-h-khan ,
Thanks for your fantastic work GeoChat, I am really interested in it. And the ckpt provided by you works for me.
However, when I tried to reproduce it as a beginner of the …
-
Two questions regarding llama 2 fine tuning:
1. it seems the prompt templates defaults to `vincuna` and cannot overwritten according to the following code:
https://github.com/lm-sys/FastChat/blob/…
-
Hi,
I have used a similar dataset as image_classification_albumentations.ipynb and reused the notbook code completely but model training failing with Target size (torch.Size([32, 224, 224, 3])) mus…
-
Hi, I'm sincerely glad that you shared your great work!
I tried to reimplement the training logic of CAV but had some troubles..
Can you take a look at what might be the problem?
train.py:
```…
-
i get the below error when i run training cell in colab FineTuning_colab.ipynb
also run cell Training parameters and all parameter parsed
No LSB modules are available.
Description: Ubuntu 20.04.…
-
Specifically, sdxl_train v.s. sdxl_train_network
I have compared the trainable params, the are the same, and the training params are the same.
As a result, batch size 10 --> 4 otherwise an gpu memor…
-
### Feature request
Currently when we enable gradient checkpointing, e.g. in `LlamaModel`, we call `torch.utils.checkpoint.checkpoint` on every `LlamaDecoderLayer`. As per [Training Deep Nets with Su…
zfang updated
7 months ago