-
It would be brilliant if we could get implemented fine-tuning methods for robust adaptation given how much better it is than LoRA and QLoRA methods.
-
To enable efficient training on GPUs and scale our repository for models with millions to billions of parameters—essential for working with large visual language models—we must implement optimization …
-
不管是baichuhan13B的qlora训练脚本还是InternLM20B的qlora训练脚本运行时都是抛出这样的错误
-
### Please check that this issue hasn't been reported before.
- [X] I searched previous [Bug Reports](https://github.com/OpenAccess-AI-Collective/axolotl/labels/bug) didn't find any similar reports.
…
-
如题,是开放出来了?对应是datasets文件夹下的哪个数据?
同理:
老母亲心理咨询师 模型 对应的训练数据是哪几个?
艾薇御姐 模型 对应的训练数据是哪几个?
感谢。
-
Hello!
The calculation accuracy of QLora training is float16, what is the calculation accuracy of qa-lora training?
My fine-tuning TechGPT-7b was successful with QLora, but using qa-lora always repo…
-
Sample:
https://github.com/intel-analytics/BigDL/blob/main/python/llm/example/GPU/QLoRA-FineTuning/alpaca-qlora/finetune_llama2_7b_arc_2_card.sh
Env:
Intel(R) Xeon(R) w7-3455
2 ARC770
ubuntu22.…
-
### Please check that this issue hasn't been reported before.
- [X] I searched previous [Bug Reports](https://github.com/OpenAccess-AI-Collective/axolotl/labels/bug) didn't find any similar reports.
…
-
Hello,
Wanted to quantize the model via awq after a merged qlora b&b nf4 mixtral moe.
the error is:
```
self._search_best_scale(self.modules[i], **layer)
File "/home/access/anaconda3/envs/s…
-
Any idea what could be causing this error?
My configuration is as follows: https://raw.githubusercontent.com/reilgun/runpod-qlora-trainer/master/scripts/run_flash_large.sh
```
Loading checkpoint sh…