-
Thanks for the great repo
i have two questions about training the models (specifically WizardCoder):
1. have you tried training with QLoRa, and not just LoRa ? are you considering adding it to t…
mrT23 updated
11 months ago
-
-
### Please check that this issue hasn't been reported before.
- [X] I searched previous [Bug Reports](https://github.com/OpenAccess-AI-Collective/axolotl/labels/bug) didn't find any similar reports.
…
-
Because of the following LLM-Leaderboard measurements, I want to perform QLoRA DPO without previous QLoRA SFT:
```
alignment-handbook/zephyr-7b-dpo-qlora: +Average: 63.51; +ARC 63.65; +HSwag …
-
Original Repository: https://github.com/ml-explore/mlx-examples/
Listing out examples from there which would be nice to have. We don't expect the models to work out the moment they are translated to …
-
Currently, we disabled Multi-GPU support for QLoRA because we didn't test it, yet. Might be worthwhile looking into this some time, so this issue is just to remember to revisit this.
rasbt updated
11 months ago
-
I am getting maximum recursion depth error after running this following command:
python qlora.py --model_name_or_path decapoda-research/llama-7b-hf
And this is the error I got:
File "/home/at…
-
### Please check that this issue hasn't been reported before.
- [X] I searched previous [Bug Reports](https://github.com/OpenAccess-AI-Collective/axolotl/labels/bug) didn't find any similar reports…
-
I have tried unsloth+qlora, it's cool and brings considerable speedup and vram reduction. But I am not sure if this repo is useful for **full fine-tune** after searching repo/website/benchmark. So can…
-
For reference, LLaMA-Factory [claims](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1/discussions/10) that using their toolkit you can QLoRA fine-tune mixtral with 28GB of VRAM.