-
Is there a way to utilize DeepSpeed for QLORA training? It looks like QLORA requires a special optimizer (e.g., paged_adamw_8bit).
-
just as title,sometimes we should use the Lora model or Qlora model, we cant't merge it with the base checkpoint
-
Hi there,
Thanks for sharing.
Any plan to support QLoRA? Please see the following paper for more information:
https://arxiv.org/abs/2305.14314
Thanks.
-
#### The inference code in `inference.ipynb` is taking 3minutes to run on Colab L4 GPU .Is their any way to speed up inference?
@swastikmaiti
-
Currently, we disabled Multi-GPU support for QLoRA because we didn't test it, yet. Might be worthwhile looking into this some time, so this issue is just to remember to revisit this.
rasbt updated
6 months ago
-
finetuning lora 方式 在v100 单卡 跑显存不够啊。据说要 用 qlora 跑。不知道你是怎么优化的,能单卡跑lora。
-
The [example of multi-gpu training on the SFTrainer docs](https://huggingface.co/docs/trl/en/sft_trainer#multi-gpu-training) shows that I should load the GPU into memory, but this doesn't work if the …
-
-
implement support for CPUs and other GPUs
-
使用qlora微调的时候,由于微调的数据过长显存溢出,能设置多卡qlora微调吗