How to use Lora in Diffutoon

[nitinmukesh]

Is there any way to use Lora

https://github.com/modelscope/DiffSynth-Studio/blob/main/examples/Diffutoon/diffutoon_toon_shading.py

[nitinmukesh]

Used it by adding load_lora. It would be better to have it as optional in config.

def load_pipeline(self, model_list, textual_inversion_folder, device, lora_alphas, controlnet_units):
        # Load models
        model_manager = ModelManager(torch_dtype=torch.float16, device=device)
        model_manager.load_models(model_list)
        model_manager.load_lora("models/lora/r1ge-AnimeRage.safetensors")

[lmpeng12]

通过添加 load_lora 来使用它。最好在 config 中将其作为可选选项。

def load_pipeline(self, model_list, textual_inversion_folder, device, lora_alphas, controlnet_units):
        # Load models
        model_manager = ModelManager(torch_dtype=torch.float16, device=device)
        model_manager.load_models(model_list)
        model_manager.load_lora("models/lora/r1ge-AnimeRage.safetensors")

You can refer to ControlNet_unit for reference

def load_pipeline(self, model_list, textual_inversion_folder, device, lora_alphas, controlnet_units,
lora_units = [],
):
        # Load models
        ...
        model_manager.load_lora(lora_units[0]['model_path'],[i]['scale'],lora_alpha=lora_units[0]['scale'] )