Please add init image support

[SoftologyPro]

Could you please merge the init code from https://github.com/talesofai/AnimateDiff with the next update? It would be great to be able to specify a seed image with this main repo code and not have to use a fork (that doees not have the latest v2 support).

[Gitterman69]

yes please! init img is a must

[gordingus]

Also interested in this feature! Was super fun to play with

[DoctorDiffusion]

Yes please! img2vid would be an outstanding addition.

[realsammyt]

great idea

[limbo0000]

We are working on that:)

[mischaschaub]

And we are waiting for it! Supergreat that you will try this, THX

[Gitterman69]

We are working on that:)

any eta? 🤞

[powerspowers]

we are working with a version that includes it but trying to get it working exactly right to take the initial image + the given prompt in combination to set the video in the right direction. Definitely a challenge! Will contribute if we get the balance right.

[ykk648]

I did some work https://github.com/ykk648/AnimateDiff

[SoftologyPro]

I did some work https://github.com/ykk648/AnimateDiff

There are already some forks that support an init image. See my first link above. Yours does look good too. I am really asking for support to be added into this main repo. That way as this goes forward and gets updated the init image changes are not lost.

[powerspowers]

@ykk648 your effort does look promising but it sounds like you ran into some fundamental block in getting it fully there in AD?

@SoftologyPro we tried a couple forks and so far none of them seemed to power using an input image as a true start frame, finding it in the latent space and then animating it by itself of with and added prompt. Best we got was using the image as the initial diffused image. Did you see anything better in one or more forks?

[SoftologyPro]

@SoftologyPro we tried a couple forks and so far none of them seemed to power using an input image as a true start frame, finding it in the latent space and then animating it by itself of with and added prompt. Best we got was using the image as the initial diffused image. Did you see anything better in one or more forks?

I didn't try any of the forks myself. I just got requests to add init image support and thought rather than supporting a different fork it would be better to have the change made to the original repo. I have added support for AnimateDiff in Visions of Chaos and a few users suggested seed image and pointed to that first repo I linked to. I tried to merge those changes in (unsuccessfully) so that is when I opened this issue.

[powerspowers]

Same here, but we could not even find the right settings for the initial diffused image to result in something reasonable as video. If you found good values and a ramp that worked I would love to know what you landed on.

Meanwhile ... if I were working with a single image and SD I would use img2img or depth2img to take an input image and stylize it or change the characters or the environment. Img2img is functional but depth2img really gives you a lot of power to keep the structure of the elements in place.

For AnimatedDiff I'm still learning what could be used get the initial image resolved in a way that AD can then animate it + what might be done to do something like depth2image where the initial image holds the structure but the prompt has a strong effect to modify it.

Searching high and low ... and learning ...

[JoshonSmith]

We are working on that:)

[TheOneTrueGuy]

It says issue is completed but I can't find anything in the docs about how to use it.