amotile / stable-diffusion-studio

An animation focused workflow frontend for Stable Diffusion
MIT License
478 stars 30 forks source link

Not all functions in-betweening #4

Closed mischaschaub closed 2 years ago

mischaschaub commented 2 years ago

Some propositions: 1) It seems that at the moment only the seed and the prompt are morphing; would be great to have this functionality for the other functions as well. 2) Would be super to include two functions for translation in Z and for a rotation around Y, this would help incredibly for animation. 3) I cannot play the rendered animation, must scroll manually in the timeline. Not essential, but would be helpful.

Am having great fun with your lovely tool. Here another small result: https://youtube.com/shorts/zNphYHW5QpQ?feature=share

amotile commented 2 years ago
  1. What other functions do you mean? Steps and CFG should work (but I will test that later). And sampler doesn't really have a "inbetween" value.
  2. I hope to add the kind of 3d animations you can do with Deforum as my next big project here, when that will happen I can't say.
  3. That's why I cut it. I was working on it but I couldn't get it "right" so I decided to release without that feature.

Thanks, and it's fun to see what people make with it. Cool video.

mischaschaub commented 2 years ago

Hi, thanks for your answers. 1)Steps and CFG do not work in my installation, so if it should not be too much to ask I would be very glad to get these two functions before your "next big project", as they have heavy influence on the results. You write correctly that samplers do not have a inbetween value. But then it would be great if we could not hook this keyframe-value up, to get a distinct break between the samplers, otherwise a late sampler choice overwrites all the former sampler choices. 2) It would be helpful if the defined keyframes could remain distinguishable from the generated in-betweens, to edit those. If one could select a whole column of keyframe-annotations and slide these as a group on the timeline. So sorry about all these requests, but you got me hooked! Are there any other people doing movies with your solution? Would be great to exchange results and experiences. It's a bit silly of me to put my results here - you got an idea or platform where to put them? For now, I add here the test of this afternoon, but I have major plans... https://youtube.com/shorts/v28FXLv9LGc Your approach is truly valid, it would be a shame to let this go - a discord group might make sense. Kind regards Mischa

amotile commented 2 years ago

1) If I can reproduce that I will fix that before the next big version yes. Samplers are supposed to switch at the 50% mark between the key frames. Will retest that. 2) These are good ideas. Will add them to my list.

The other place I've seen someone use it is on the original reddit post:

https://www.reddit.com/r/StableDiffusion/comments/xyriur/comment/is26fto/?utm_source=share&utm_medium=web2x&context=3

If you make cool things like this, I think it's a good idea to share that and maybe how you made them to drum up more interest.

A discord is probably premature, but what do I know :D

mischaschaub commented 2 years ago

OK, will communicate about your solution with some samples and inform you about my postings. The main remaining problem is that the app hangs from time to time with a sand watch symbol over all frames, no more rendering possible. Is there any magic solution for this? At the moment only a full restart seems to help, what sometimes is a shame and always a nuisance.

amotile commented 2 years ago

The backend? Or the browser tab? Either way I haven't seen it so I don't have a solution at the moment.

mischaschaub commented 2 years ago

In the browser. Good to know, then the mistake may be in my installation.

amotile commented 2 years ago

I'll keep an eye out :)

amotile commented 2 years ago

Steps and Cfg seem to be working for me: image image

Could you get a screenshot how it looks for you? I was wrong about samplers they stay the same until a new key frame.

amotile commented 2 years ago

I didn't fix that one but the new release has a few of your suggestions

mischaschaub commented 2 years ago

Just a quick feedback, must go for a walk through the forest here in France to collect mushrooms. Now the steps are variable. great. What still disturbs is the superlarge negative prompts window positioning itself in top of important info, lower and shorter would be better, or at least it should be movable. And it pops up with its offer for negative prompts, when I want to change the normal prompt - so perhaps the window should have two fields to edit for the positive and the negative prompts. Will give you more feedback during the weekend. Many thanks for your superfast service!

2022-10-14 06 33 01