search

Kosinkadink / ComfyUI-Advanced-ControlNet

ControlNet scheduling and masking nodes with sliding context support
GNU General Public License v3.0
619 stars 61 forks source link

Question [NOT A BUG] #33

Closed NullEqualsZero closed 11 months ago

NullEqualsZero commented 11 months ago

Sorry to bother you, but I can't find the info I need. I mainly like to do Vid2Vid with animate diff (in ComfyUI). I am trying to change both the strength and the ending step of controlnet for like 2/3 of my frames but I don't know what nodes I should use. I was thinking about the laten batch thingy but I am just not sure. This is what I have come up with but I am not sure it is correct any help would be most welcome. Also following the readme I cant find some of the nodes that you are talking about (the don't have the same inputs and before you ask I am on the latest update) image

Kosinkadink commented 11 months ago

The reason the inputs aren't there is because when you load a workflow that uses an old version of the workflow, you have to 1) reload your webpage so comfy can put in the missing parameters, and 2) for some parameters, you may need to just delete the node from your workflow and place a new one. Unfortunately, it's just the way JavaScript stuff works with comfy. And yep, you've got the right idea. I can post a screenshot of some stuff in a sec when I get to my PC, in the meantime see if you can have the nodes appear fully.

NullEqualsZero commented 11 months ago

Thank you, man. My issue with understanding this is the following first I am on ComfyUIworkflows.com which has the node/extensinon automagically downloaded so it has to be updated by them and I asked if it is on the latest reales. the other thing I have is that even with a fresh workflow I can get this it only has the time steps input on it. that's why I am using it. Also, how would I change my end present? image image

NullEqualsZero commented 11 months ago

I guess that on the site it is not up to date since on my local install (I am currently not using it due to very low VRAM 4gb (PAIN)) it shows the nodes correctly.

Kosinkadink commented 11 months ago

If you refresh your webpage (or open up a new tab), it will pull the proper .js, and manually put it in.

image

Here, starting at the beginning (start_percent 0.0), you can use Latent Keyframes to set the strength of the controlnet in those frames. By default, null_latent_kf_strength is set to 0.0 due to this being the default before, but in your case you will need to set this to 1.0 so that the other frames that you don't explicitly set the strength of will default to that strength - I think i will refactor the code so that will default to 1.0, but I'll have to think it over.

Anyway, the settings on that Timestep Keyframe will keep applying until 1) the Apply ControlNet node reaches the end percent, or 2) there is another Timestep Keyframe with a later start_percent. In the example image provided, since start_percent on the other Timestep Keyframe is 0.8, 80% of the wya through sampling, it will switch to that Timestep Keyframe. By applying a different set of Latent Keyframes there that have different strengths, the remainder of the ControlNet sampling will use those new strengths.

NullEqualsZero commented 11 months ago

thank you so much trying it out now I will keep you updated on how it goes