s9roll7 / ebsynth_utility

AUTOMATIC1111 UI extension for creating videos using img2img and ebsynth.
1.25k stars 128 forks source link

Analyse optical flow in original video #63

Open Karleener opened 1 year ago

Karleener commented 1 year ago

Hi. Thank you for your very good job simplifying the use of EbSynth with Stable Diffusion ! I published a tutorial (in french) here https://www.youtube.com/watch?v=-tQ-P4TFz3M I have some questions Is it possible to disable the mask computation in the first stage ? I don't use this feature and it can be a quite long process. Is it possible to ask some confirmation before to delete the img2img_key directory content? I mean that sometimes if you are not satisfied of the content for one frame, you test a prompt in img2img, and if you forgot to check the box "TEST", then the full directory content is deleted without any warning.

Possible improvements : Due to the computation performed by EbSynth, it is necessary to animate each plan of a video separately. A change of plan add some visible artefacts during the flow analysis / pixel synthesis. I animated a sequence containing 17 plans, and I had to create manually 17 sub-directories, 17 videos, 17 animations. I would be nice to automatize this process, detecting plan change in the original video, and create automatically the 17 sub directories. An simple analyse of flow motion in each plan could allow also the parameters of the stage 2 (subsampling of original frames) to be tuned automatically. Do you plan to add such features? I can collaborate for image processing using Opencv. Best Karleener

s9roll7 commented 1 year ago

Thanks for the suggestion

Is it possible to disable the mask computation in the first stage ? I don't use this feature and it can be a quite long process.

You can disable the mask mode from the following menu Ebsynth utility -> configuration -> etc -> Mask Mode

Possible improvements :

I felt have reached the limit of video generation using ebsynth and will probably not make any major changes. Instead, I am working on an extension that will generate videos using a different method, and I was spending time on that.

https://github.com/s9roll7/sd_loopback_music_sync_wave

Karleener commented 1 year ago

Thank you, the new method seems promising. Perhaps faster than EbSynth ? With EbSynth, I obtained the following results https://www.youtube.com/watch?v=b-FQLwdNdwU