search

volotat / SD-CN-Animation

This script allows to automate video stylization task using StableDiffusion and ControlNet.
MIT License
806 stars 61 forks source link

TWISTING & EXTREME WAVYNESS #151

Open Johnnyprim3 opened 1 year ago

Johnnyprim3 commented 1 year ago

Hi Brother, once again, thanks for such a wonderful tool & the updates you keep making. However, I don't know if it's the result of an update or some tweaking but now, when you try to generate a video, it starts out okay but warps after the 6th to 8th frame. I too thought it was my perception playing tricks on my eyes but, I'm one of the biggest users of your extension & have generated uncountable videos & this problem was non-existent. Sure it would be wavy but it was consistent, now it just waves away into a smeary mess. Secondly, I couldn't help but notice the 'Extra settings' option that appears in Txt 2 video & disappears when you attempt to update it...I imagine this option would help regulate the extreme wavyness...it exists in Vid 2 vid but I wonder if it'll affect Text 2 Vid? Something has changed and it's affecting the output from this beautiful extension. Thanks for your time Brother, Cheers.

Gitterman69 commented 1 year ago

i have the exact same problem... txt2vid doesnt work at all imho

https://github.com/volotat/SD-CN-Animation/assets/113683801/ee9b8e1a-8d5e-48ff-a53d-442063947cd4

https://github.com/volotat/SD-CN-Animation/assets/113683801/26b84312-e92e-4a8f-8d99-88e1e9ac2d72

thezveroboy commented 1 year ago

i noticed the same problem here https://github.com/volotat/SD-CN-Animation/issues/135

Ainaemaet commented 1 year ago

Same issue here as well, tried doing a long gen last night for the first time in a couple weeks and woke up to 2 and a half minutes of what looks like one of those sand in water things.

thezveroboy commented 1 year ago

if you want to get normal generations you need to downgrade to the version 0.6. all versions after has been broken

Johnnyprim3 commented 1 year ago

if you want to get normal generations you need to downgrade to the version 0.6. all versions after has been broken

any tips on how to do this?

thezveroboy commented 1 year ago

if you want to get normal generations you need to downgrade to the version 0.6. all versions after has been broken

any tips on how to do this?

just use this https://github.com/volotat/SD-CN-Animation/releases/tag/v0.6

Johnnyprim3 commented 1 year ago

if you want to get normal generations you need to downgrade to the version 0.6. all versions after has been broken

any tips on how to do this?

just use this https://github.com/volotat/SD-CN-Animation/releases/tag/v0.6

Thanks Brother, but when I pasted this into the extensions folder and ran A1111, it doesn't show the text box for the text 2 video prompt. Any suggestions

VickyHax commented 1 year ago

if you want to get normal generations you need to downgrade to the version 0.6. all versions after has been broken

any tips on how to do this?

just use this https://github.com/volotat/SD-CN-Animation/releases/tag/v0.6

Thanks Brother, but when I pasted this into the extensions folder and ran A1111, it doesn't show the text box for the text 2 video prompt. Any suggestions

me too! Any help would be great on getting a NON SQUIGGLY version of this awesome extension! thanks! <3

rbfussell commented 1 year ago

https://github.com/volotat/SD-CN-Animation/issues/135#issuecomment-1622124515

'About a month late, but looking at the occlusion image, It appears to be an issue with the the processing of the occlusion image. As you can see here in this video of the occlusion images, the edges of the image moves in the direction of the shearing of the actual image contents. Those white spaces on the edges, are not present in the actual image that is displayed.

https://www.youtube.com/shorts/_c1mjOcyq1s

as can be seen here (sorry it is a youtube short, youtube doesn't give an option to make it a normal video due to size and length)

It appears to be occuring in the flow computions within RAFT I assume. I am not sure how RAFT works, but I feel that there needs to be a bounding box that it cannot simply "peel" the image away from leaving the white "blank" areas. This MAY not actually be the issue, however, and instead it may simply be the flow calculations themselves.'

I think the fix may be as simple as forcing it to stretch the edges instead of "peeling" back to show no image as white in the occlusion mask image, as it is doing. This could lead to the weird calculations that ends up making the later image generation such garbage.,