-
### System Info
8xH100
### Who can help?
_No response_
### Information
- [ ] The official example scripts
- [ ] My own modified scripts
### Tasks
- [ ] An officially supported ta…
-
Hello Expert,
an error happened when export gguf with F16.
>>> **model.save_pretrained_gguf("model", tokenizer, quantization_method = "f16")**
Unsloth: Merging 4bit and LoRA weights to 16bit…
-
Hi!
I am using mistralai/Mistral-7B-v0.1 and when I set CUDA_VISIBLE_DEVICES=1 (or anything else than 0) it gives me the error below. After checking I think that the culprit is this [line](https://…
J4Q8 updated
5 months ago
-
Hello there,
I am trying to use fine-tuning using the SFT Trainer, but when starting the training process when calling `trainer.train()` my system throws a Segmentation Fault.
My system is equipp…
-
### Reminder
- [X] I have read the README and searched the existing issues.
### System Info
python 3.10
vllm:latest
### Reproduction
do you have any guide or documentation on how to incoporate…
-
I use this code to fine-tune Llama-3-8B-Instruct (with unsloth):
```python
from unsloth import FastLanguageModel
import torch
from trl import SFTTrainer
from transformers import TrainingArguments…
-
I was trainging Gemma-2B and Gemma-7B using sfttrainer, with `packing=True` set. The Gemma-2B's loss was quite normal, but Gemma-7B's was abnormally high. I'm not sure why this would happen, since bot…
-
Hi there, I finetuned Mistral 12b Nemo using unsloth. However, when downloading my LORA adapters from hugging face, to use this model to generate text, I get this error:
"ValueError: The following …
-
I am trying to generate batch data (greater than 100w pairs), but i find the speed of model inference is slow, any advice?
```
import torch
from unsloth import FastLanguageModel
model, tokenizer =…
-
### Reminder
- [X] I have read the README and searched the existing issues.
### System Info
- `llamafactory` version: 0.7.2.dev0
- Platform: Linux-4.18.0-517.el8.x86_64-x86_64-with-glibc2.28
- Py…