-
Hi there,
Thanks for sharing.
Any plan to support QLoRA? Please see the following paper for more information:
https://arxiv.org/abs/2305.14314
Thanks.
-
Currently, `tune ls` is a bit unweildy. Can we make it better?
@joecummings
-
The training process is quite slow, whereas using 8-bit hqq speeds it up by more than tenfold. Is this normal? Or have I missed any code?
```python
import torch
from transformers import EetqConfi…
-
### Please check that this issue hasn't been reported before.
- [X] I searched previous [Bug Reports](https://github.com/OpenAccess-AI-Collective/axolotl/labels/bug) didn't find any similar reports…
-
Because of the following LLM-Leaderboard measurements, I want to perform QLoRA DPO without previous QLoRA SFT:
```
alignment-handbook/zephyr-7b-dpo-qlora: +Average: 63.51; +ARC 63.65; +HSwag …
-
I had run pretty much this exact command a couple weeks ago when doing benchmarking but now it is failing with a stride mismatch error. Creating this issue so others can take a look as well. Repro and…
-
Traceback (most recent call last):
File "./train_qlora.py", line 235, in
main()
File "./train_qlora.py", line 224, in main
train_result = trainer.train()
File "/usr/local/lib/pytho…
-
i have successfully fine-tuned the model using QLORA for a custom use case. now i have the LoRA adapters and can you tell how to use it for the inference. maybe merge lora weights with the original mo…
-
Hello everyone.
When I trained Mixtral-MOE with QLoRA + Zero3, it occurs error like below.
…
-
Thanks for the great repo
i have two questions about training the models (specifically WizardCoder):
1. have you tried training with QLoRa, and not just LoRa ? are you considering adding it to t…
mrT23 updated
9 months ago