search

XmYx / deforum-comfy-nodes

Deforum ComfyUI Nodes - ai animation node package
https://deforum.studio/
MIT License
152 stars 19 forks source link

How to use LoRa #45

Open ShylockGao opened 3 months ago

ShylockGao commented 3 months ago

Could you please tell me how to integrate ControlNet、LORA? Also, could you show me the workflow?

XmYx commented 3 months ago

Heya. for LoRA loading, please refer to normal comfyui guides, but in practice, you'd place a Lora Loader node after the Checkpoint loader, and route your clip and unet (model) through it, and into the original nodes in the workflow (text encoding, sampling.). For ControlNet with deforum, you'd want to use a (deforum) Load Video node, which returns a single frame on each execution, or use logic nodes to control other video loader nodes to return the relevant frames desired for your animation, then route that into a ControlNetApply node which normally sits before the sampler, after the text encoder, as the information get's embedded with the conditioning each time.

https://www.runcomfy.com/comfyui-nodes/deforum-comfy-nodes/DeforumControlNetApply

https://www.runcomfy.com/comfyui-nodes/deforum-comfy-nodes