search

vladmandic / sd-extension-steps-animation

Save Interim Steps as Animation extension for SD WebUI
143 stars 19 forks source link

Final image is not included in animation #32

Closed DasWookie closed 1 year ago

DasWookie commented 1 year ago

If the final image is run through post processing, for example "Restore Faces", this final image result is not included in the video sequence.

vladmandic commented 1 year ago

true as that happens after script has alyready finished. it would be possible to capture it using a completely different callback, but there is no way to know if final step is equal to final image or something postprocessed it. so if there is no restore faces, it would be a duplicate. also, what if postprocess is hires fix or upscale? then the final would have completely different resolution.

it simply makes no sense to capture it.

DasWookie commented 1 year ago

While I completely agree that resolution changes would make no sense, but Restore Faces is very much worth capturing as this can have a significant impact on the final image. It seems like checking if the final image is different (md5 / hash perhaps) or not wouldn't be too difficult to do as a check, or perhaps a check for if the UI had Restore Faces checked or not. Considering this is one of the default options for image modification, which is used very frequently, it could be very useful. I know that the generated images I create without Restore Faces frequently aren't worth consideration of keeping, where as when Restore Faces is used, it can have a pretty marked improvement.

vladmandic commented 1 year ago

unfortunately, there is no way for a script to know if restore faces or something else is enabled - I would have to import main UI and check if checkbox is selected, but that is really ugly.

off-topic, restore faces was very commonly used in early days while default model was really poor with faces. but with tons of good models available nowadays, none of the artists i know use restore faces anymore.

DasWookie commented 1 year ago

While faces have vastly improved from the initial trainwrecks over the last year, it still does wonders for fixing eyes.

I'm asking for a compromise here I realize. No solution is going to be perfect, but an attempt at grabbing the final image to include has real value.

Maybe a radio button to elect to try and make a best effort at including the final image. This way, if something like fix faces is used, it would grab it and include it. If not, it works as currently. If somebody checked it and there was no change they yeah, you'd get a duplicate of the unmodified image. If it was checked and some other process changed it with sizes then well, yeah ffmepg is going to likely have a fit, but it's at least a choice.

I'm trying to create some videos right now for a presentation, and the final image with fix faces is VERY much making a big difference. I also plan to do some live demonstration of SD capabilities, and initial feedback from a few of the target audience are interested in seeing the in-process steps as well because there's some overlap between another plugin with CFG Scale and they want to be able to specifically see how CFG Scale is impacting the final result and the video creation is a perfect way of using the two extensions together.

vladmandic commented 1 year ago

i'll reopen, but given my load, not likely to be implemented soon.

KominoStyle commented 1 year ago

true as that happens after script has alyready finished. it would be possible to capture it using a completely different callback, but there is no way to know if final step is equal to final image or something postprocessed it. so if there is no restore faces, it would be a duplicate. also, what if postprocess is hires fix or upscale? then the final would have completely different resolution.

it simply makes no sense to capture it.

why not making another step video when hires fix starts?

vladmandic commented 1 year ago

i've looked at this and capturing image that happens in postprocessing from inside extension is possible, but messy. and my initial comments on why this is mostly not a good idea still hold. yes, use case for face restore is valid, but as it is, i don't plan to add this. if someone contributes a pr, i'd be wiling to look at it.