-
Hello, thank you for the amazing work, is it possible to use Qlora to fine tune the 4bit quant models?
-
Can I load QLoRA fine-tuning weights into a Hugging Face model as shown below?
```python
model_id = "meta-llama/Meta-Llama-3-8B-Instruct"
quantization_config = BitsAndBytesConfig(
load_in_4bit=T…
-
Trying to finetune qwen2 7b bnb 4bit
getting below error.
if_bad_first = any(x in chat_template for x in actual_bad_tokens)
TypeError: 'in ' requires string as left operand, not NoneType
the sam…
-
Does Simpo support Qlora fine-tuning using 4bit for Llama-3-8B ?
if so, how to set it up?
-
### ⚠️ Please check that this feature request hasn't been suggested before.
- [X] I searched previous [Ideas in Discussions](https://github.com/OpenAccess-AI-Collective/axolotl/discussions/categories…
-
Can you provide a detailed guide on how to use LoRA or QLoRA finetuning of VideoLLaMA2 and also for inference. Moreoever, can you provide the guide on how to make the finetuning dataset for images , …
-
Hi, I noticed in the tech report of LLama3-8B-80K that, the authors evaluate the vanilla LLama-8K-Instruct in the LongBench dataset with 8K context length, and obtain the following results:
![image](…
-
Hi, first of all thanks for the great tutorials on lora and qlora! I was able to follow them very easily.
I was wondering if multi-gpu QLoRA is supported? I couldn't find a config file in the repo, a…
-
### 🚀 The feature, motivation and pitch
Support tensor-parallelism in QLoRA on vllm.
### Alternatives
_No response_
### Additional context
_No response_
-