search

Zeju1997 / oft

Official implementation of "Controlling Text-to-Image Diffusion by Orthogonal Finetuning".
https://oft.wyliu.com/
MIT License
280 stars 14 forks source link

Have you tried training OFT on the SDXL #15

Closed carvychen closed 7 months ago

carvychen commented 10 months ago

If so, could you please also release the relevant training code?

Zeju1997 commented 10 months ago

Hi, thanks for the question. We did not test it on SDXL yet, but might add additional experiments of SDXL in the future. Best,

Zeju1997 commented 7 months ago

OFT has been integrated into the PEFT repository (https://github.com/huggingface/peft). Here is the guide of finetuning dreamooth with PEFT-LoRA: https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/README_sdxl.md By changing the LoRAConfig to OFTConfig, finetuning SDXL can be easily achieved.