Timestep Keyframe-ing `question`

[NullEqualsZero]

I am having trouble understanding how to do something very specific. I want to make a video using Animatediff in ComfyUI (basically a video-to-video). However, ideally, I would like to have a control net (Lineart) change strength at two different percentages of the way done( I don't know how to say this properly). Basically, I want to begin with it having about 0.2 strength for the first 10% of the way and then switch to having something closer to 0.8 strength till the 0.65 step percent. (I Know that my terminology is not correct if you need any clarifications I will gladly provide them). The video doesn't have a set amount of latents so that's also something that is confusing me on how to approach this. #33 (You have helped me before but I could not give it a try due to GPU limitations)

PS: All edits were made in the hopes of trying to find how to label the issue. I then found out that I couldn't do that😭

[Kosinkadink]

First things first, are you asking about wanting to change controlnet strength at specific TIMESTEPS (i.e. at step 5 or at 0.25 (25%) of the sampling steps, make the strength 0.67), or are you asking about controlling controlnet strength at specific LATENTS/FRAMES (i.e. at the 10th frame of the animation, make the strength 0.45 (45%) strength)?

If it's about TIMESTEPS, then the amount of latents does not matter at all. The amount of latents only matters for LATENT (frame) keyframing. They can also be used in combination with each other, but we can get to that when you clarify which exactly you're trying to do (also, you can reference the graph I made in the readme to see if that helps explain what exactly you intend to do).

[NullEqualsZero]

Yeah sorry for my lack of knowledge and wrong use of terminology. The answer is that I want to do the timestep thing (at least for now)

[NullEqualsZero]

however, if you could teach on how to do both I would be more than happy.

[Kosinkadink]

Here are two ways to do the behavior you asked about in your first comment (0.2 strength until 0.1 of the way through sampling, then 0.8 strength until 0.65 of the way through sampling, and then I wasn't sure if you wanted it to be 0.0 strength afterwards or 1.0. The example is for making the strength be 0.0 after 0.65 of the way through sampling.

The top approach sets the end_percent for the controlnet at 0.65, so the entire controlnet stops being used after 0.65 of the way through sampling, achieving the 0.0 strength after 0.65 by not running anymore.

The bottom approach sets strength at 0.65 to 0.0 explicitly, and my code will interpret these two approaches as basically hte same - if strength is set to 0.0 on either the Apply node or a Timestep Keyframe, it will not do anything for the sampling steps at which it is zero strength (making sampling faster).

The strength of a Timestep Keyframe is multiplied by the strength of the Apply ControlNet node, so keeping Apply ControlNet at strength 1.0 will make the strengths of the Timestep Keyframes be the effective values, and you can then use the strength of the Apply ControlNet as a multiplier if you want to weaken the overall effect.

I will rework Timestep Keyframes in the near future to work more inline with my Context Options and AnimateDiff scheduling, but for the time being, Timestep Keyframes set what strength the controlnets should be multiplied by at a given relative start_percent. Hence, in both the top and bottom approach, at start_percent 0.0, strength is set to 0.2, then at start_percent 0.1, the strength from the Timestep Keyframes will instead be replaced by 0.8, and then at start_percent 0.65, it will become strength 0.0.

[NullEqualsZero]

Thank you that exactly what i was looking for

[NullEqualsZero]

What is null_laten_kf_strength? What does it do?

[Kosinkadink]

null_latent_kf_strength is only relevant with the use of latent_keyframes. null_latent_kf_strength is the strength that should be given to any unspecified Latent Keyframes strengths (i.e. when you have 32 frames in your animation, you provide strengths for only frames 5-18; the null_latent_kf_strength is the strength that will be given to the unspecified frames).