-
Hello,
I've ran a full finetuning on Flux using guidance_scale=1.0.
Now when I try to inference the model, I need to set the CFG scale > 1 because if I let it at 1 to disable it like I'm used t…
-
First, would like to thanks ostris for this amazing tool, after tried Kohya and Simpletuner, ai-tool kit give me better results with a great ease. Would like to know if they're is a plan for create a …
-
Following [how we do it in LoRA recipes](https://github.com/pytorch/torchtune/blob/afd23fd0b2f9051958affae20890396e2594756f/recipes/lora_finetune_distributed.py#L475), we should add the ability to use…
-
As in the title.. I spent a bit of time debugging it but haven't figured out the cause yet. E.g. running
```
tune run --nproc_per_node 2 full_finetune_distributed --config llama2/7B_full fsdp_cpu_…
-
### Reminder
- [X] I have read the README and searched the existing issues.
### System Info
- `llamafactory` version: 0.8.3.dev0
- Platform: Linux-6.5.0-35-generic-x86_64-with-glibc2.35
- P…
-
## TODOs
- [ ] Fix the speaker embedding finetuning code https://github.com/lenML/ChatTTS-Forge/blob/318d33f8d0b1451a39b3cbc94debca7f4f21dfca/modules/finetune/train_speaker.py#L15-L26
- [ ] Use the …
-
In the full_finetune_distributed (https://github.com/pytorch/torchtune/blob/d31649e75e51cf15dbdca004288c32550e8f7f8d/recipes/full_finetune_distributed.py#L428C32-L428C41), we call init_cache before tr…
-
### Issue Description
LoKr on flux isn't working.
Example LoKr: https://civitai.com/models/714292
Basically LoKr allows for full finetune like experiences, without the full vram costs of full f…
-
# Alex Strick van Linschoten - My finetuned models beat OpenAI’s GPT-4
Finetunes of Mistral, Llama3 and Solar LLMs are more accurate for my test data than OpenAI’s models.
[https://mlops.systems/pos…
-
Hello, I'm interested in knowing if there are any plans to implement Full support for Direct Preference Optimization (DPO) in the upcoming releases.
Are there any current efforts or roadmap items r…