Problem with color gradual changes for long videos

[xuxu116]

Thanks for the release code! I have experiencd severe color changes for long video generation, e.g more than 60 frames. And finally the color becomes oversaturated. Tune off the init_noise_by_residual_thres can alleviate this issue, but the first frame conditioning still makes the color changing aorund.

Another problem is that the unet3d fails to generate images without control_hit, which is not expected as the normal unet2d with controlnet.

[pravdomil]

similar problem here

https://github.com/Weifeng-Chen/control-a-video/assets/2387356/3a131bf6-ab33-4138-820c-40aa67766d34

[Weifeng-Chen]

yeap. the algorithm is not stable in this time. in my experment, it will collasp in 3~4 rounds