genialgenteel
commented
3 months ago Agree, and I don't believe this has been fixed yet from my testing. Last year some folks were discussing how img2gif is broken here: https://github.com/continue-revolution/sd-webui-animatediff/issues/96, but I don't believe any universal solution was reached. The proposed solution of raising denoising (seems counterintuitive given you'd want your first frame to be very similar if not identical to your input, right?) and unchecking "Always discard next-to-last sigma" did not work for me, and I left a comment there as well mentioning as much.
There was one person in that thread who proposed a (hacky) solution, but I don't think it was ever implemented into AnimateDiff.
It's unfortunate that half a year later this issue has not been solved still. I'm no programmer, so I can't figure it out myself. I guess what we're really asking for is a zero-shot image animator in the vein of AnimateZero, but that code hasn't been released yet (and at this rate I'm not sure it ever will!).
Expected behavior
Expectation: You should be able to have the option to gradually increase denoise value for img2gif, as a better method than messing with the init_latent
Explanation: Right now img2gif seems to basically function (from what I can tell) by feeding each frame a progressively "messier" version of the img.
(I simply tested this by setting the denoise value very low and running img2gif, for which the resulting gif fades from the initial picture to a grey mess)
(I might have some details wrong because I don't know much about latents, but I'm trying to base this purely on what I can observe)
This method of turning the initial picture into a grey mess works fine if the denoise strength setting is set high enough that the diffusion process can turn that grey mess around the last few frames into a good frame.
(I assume the intention is to get the start of the gif to be based on the initial image and the destroying that initial image with mess allows the rest of the gif to deviate from the initial image)
However if you have the denoise set high then the first frame of the gif with not be very close/similar to your input image, and if you try to lower the denoise strength such that the starting frame is very close to your input image then you will end up with either a very static gif or one that end up as a grey mess (depending on how you set your latent power and latent scale).