volotat / SD-CN-Animation

This script allows to automate video stylization task using StableDiffusion and ControlNet.
MIT License
807 stars 61 forks source link

This is for the Creator... #180

Open Johnnyprim3 opened 11 months ago

Johnnyprim3 commented 11 months ago

I'm going to be very blunt: Your Work is a gift. The BEST Text2Vid generator, the MOST versatile and that allows for INFINITE creative options. Is it perfect? Nope. However it is REMARKABLY smart in creating continuous movement almost scarily remarkable motion. I understand the amount of things you'd have to juggle to make all this come together. Hang in there. Your work is appreciated and makes me Challenge my own creativity. We trust your judgement as far as the future of this generator is concerned. That's it, friend. You're appreciated. cheers.

Ainaemaet commented 11 months ago

I really do hope he's ok; Indeed this extension is amazing - and one of the only ones that you can load and use nearly any model for generation. Super unique and haven't found anything like it otherwise!

IanD-FM commented 11 months ago

FYI he posted something on twitter on July 25 (link is in his profile) so I think he is ok. Volotat if you're reading this, please come back! We appreciate you!

Ainaemaet commented 11 months ago

I seen that; knowing that he's ok it's certainly a bummer that someone would let this become abandonware.

Perhaps his thinking is that things are developing so rapidly we all know we are going to have even better software fairly quickly? The thing is, what this does is entirely novel!!!

volotat commented 11 months ago

Hello. Thank you for the warm words. I'm too busy right now with my job and some personal stuff so I have little to no time working on this project. It was mostly built on my vacation when I had free time to spare. Also I was quite overwhelmed with works like this: https://anonymous-31415926.github.io/ and https://github.com/guoyww/AnimateDiff that made methods used in this project simply outdated. Then we had extremely good development on the text2vid methods, such like Zeroscope-v2 model that could also do vid2vid pretty well and has dedicated extension https://github.com/kabachuha/sd-webui-text2video. So it seems to me that SD-CN-Animation needs to go, as I don't really see anything of value that I could add to the field at this point.

peteTater commented 11 months ago

@volotat Thanks for the update. This is sad news man.

Life has gotten in the way for me over the last few weeks but I still intend to share the animations I've been creating over in Discussions when time permits.

While the other work you point to certainly holds promise, Rerender A Video doesn't exist yet in any practical, useable format, AnimateDiff currently seems to be exclusively for the 3090ers and Zeroscope-v2 renders a max of 125 frames on a 12gb VRAM card.

Your offering blows all of these out the water in terms of versatility and useabilty on lesser than flagship cards, and so far only Deforum compares, although it produces distinctly different results, for high res anims at extended lengths (once I'd changed that hardcoded "2048" frame length in your script to uncap the vid lengths).

Personally I think you're underestimating the value that the txt2vid tab of this extension has added for animators. In three months I've made more than a feature length's-worth of high quality scenes using the extension to use as source in my Unity interactive works, a multichannel installation, and there's a couple short films in development that wouldn't be if not for the great gift of your extension.

Creating that amount of work in so many different styles would have previously taken animators a lifetime to achieve. I've sunk 15+ years in to my craft and this tool has been like a tidal wave. There's so many experimental project ideas I wouldn't previously have invested the energy into achieving because of the time involved, but now I'm able to try out new things daily and have results I'd be happy to put my name to within a few hours of model testing/prompt tuning/param tweaking.

I know the AI world is moving absurdly fast but to say these methods are outdated because the bloodiest of bleeding edge approaches have made a splash in the last few weeks is seriously selling your own contributions short in this field! You've created something truly unique.

Perhaps striving for different outcomes as the tool's developer than some of the creators working with it, has resulted in you looking at some shiny new apples and left you feeling like this juicy orange won't now become the golden apple you had envisioned, but I'd be surprised if anything comes along for a long time that can truly compare to the functionality this extension offers creators because of its design and VRAM friendly approach. Especially to those of us who remain in the meagre sub-3090 realms.

Thanks again for this amazing tool, and as someone who has been having a great time using it, I sincerely hope that if you've decided to formally pass the baton, someone with the knowledge comes along soon and decides to pick up where you left off. You took us far.

Lots of love and hope life's treating you well.

vapidvim commented 11 months ago

@volotat, if you create a Patreon I would subscribe and sure many others would too. Maybe the community support and passive income would motivate you to continue when you have spare time? The txt to vid is unique and interesting, there is nothing else like it. Though it can be finicky, when you get a good working prompt it's totally worth it.

I really appreciate the work you put into this and will be using my animations for an exhibit in October.

oxencorp commented 11 months ago

Count me in with those that are disappointed with the end of development on this- it's the only thing I've found that can create high quality high res animation of almost endless variety and duration. I would happily subscribe to a Patreon to support basic development of this code, keeping it up to date, compatible and adding batch processing. Just so you know- for now I intend to keep a machine dedicated to running this plugin without doing a bunch of updating and risking breaking the functionality. That's how much I care about this code. Still I understand you have no obligation to support this and other peoples passions are not necessarily our own. Glad to hear you're ok in any case!

Johnnyprim3 commented 11 months ago

I am going to build something with this extension that should establish the Awe in everyone's minds. It's a gift we were given...the 80 Trillion dollar question is: Now YOU have it...what are YOU going to CREATE with it?