search

microsoft / unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
https://aka.ms/GeneralAI
MIT License
19.14k stars 2.44k forks source link

LoRA for KOSMOS-G #1475

Open janzd opened 4 months ago

janzd commented 4 months ago

The KOSMOS-G paper and the project page mention using LoRA for image generation with KOSMOS-G. How and in what format can we add trained LoRAs to the base SD model of KOSMOS-G? I couldn't really find out how to do it from the code.

xichenpan commented 3 months ago

@janzd Hi, you can refer to the code here https://github.com/microsoft/unilm/blob/b824f95357f63cafbd7bfd7ed73c2f7f49b8254d/kosmos-g/app_model.py#L108