search

continue-revolution / sd-webui-animatediff

AnimateDiff for AUTOMATIC1111 Stable Diffusion WebUI
Other
3.11k stars 258 forks source link

[Question]: How do I perform animatediff tile-upscale #232

Open ShmuelRonen opened 1 year ago

ShmuelRonen commented 1 year ago

Expected behavior

Hello, and thank you for you work!!

I work with animatediff-cli-prompt-travel and there is the animatediff tile-upscale option. How do I implement it here?

Example: animatediff tile-upscale "W:\animatediff-cli-prompt-travel\output\2023-10-19T19-20-05-anime-incredibleworld_v20\00-2529226679637802235" -c config/prompts/anime.json -W768 -H1152

continue-revolution commented 1 year ago

Try using ControlNet tile directly with source video frame path as control images (put the path to frames into AnimateDiff video path or ControlNet batch tab), if that’s gonna work then that is tile-upscale. If that’s not gonna work, we will have to read the source code.

ShmuelRonen commented 1 year ago

I will check it. Is it possible to put the PNG files of each video in a separate folder and not have them all together, for example, in the outputs\txt2img-images\2023-10-19 folder?

continue-revolution commented 1 year ago

Go to Settings/AnimateDiff and you will know which setting you should check

ShmuelRonen commented 1 year ago

Thanks! now the PNG files is in a separete folders. When I do an upscale in animatediff-cli-prompt-travel, Besides the PNG files, it uses the creation script (json) to create the upscale with tile-upscale. How can I solve it in sd-webui-animatediff?

Example: prompt.json

continue-revolution commented 1 year ago

I don't think this config has anything else you need to config in webui. Just use ControlNet tile directly and see what's going on.

ShmuelRonen commented 1 year ago

I will try and update you.

ShmuelRonen commented 1 year ago

OK. As you wrote, I try using ControlNet tile directly with source video frame path as control images (put the path to frames into AnimateDiff video path or ControlNet batch tab), if that’s gonna work then that is tile-upscale. The original video was 512x512 and I changed to 1024x1024. it's works for me but take a very long time (25 min for 16 frames) on my RTX 3090. Original: 01235-3229817589 Tiled: 01236-359361637

Now I try to do tiled-upscale on animatediff-cli-prompt-travel with the same PNG files. now it's take 2 min only for the same resoults.!!! Tiled: 00_5667687773227342612_Alberto-Vargas-1990s-East-LA-chola_1990s-East-LA-c

Can I make it better?

continue-revolution commented 1 year ago

Thanks for your report. I will investigate.

ShmuelRonen commented 1 year ago

You should use control_v11f1e_sd15_tile and not control_v11u_sd15_tile !!!

davedaverave commented 1 year ago

What's the difference between the first tiled and second tiled upscale video? how were able to make it render in less time??

ShmuelRonen commented 1 year ago

control_v11u_sd15_tile is an older version and control_v11f1e_sd15_tile is the new one.

Example control_v11f1e_sd15_tile: 01236-359361637

Example control_v11u_sd15_tile:

01238-4260294787

The processing time taken by both is pretty much the same.

ShmuelRonen commented 1 year ago

I noticed that if the tiled-upscaling process does not use the original prompt, the proposal comes out distorted. Also in the mode of prompt-travel.

I don send you an example because is NSFW...

ShmuelRonen commented 1 year ago

OK here it with cover... compatibility of taild-upscale to the prompt.

Test No 1, upscale with automatic 1111 controlnet tile-upscale. it's take 17 min and the prompt commands are not kept.

https://github.com/continue-revolution/sd-webui-animatediff/assets/80190186/df95eeec-587b-4459-a5b0-fd664e7bf73a

Test No 2, upscale with animatediff-cli-pt tile-upscale. it's take 3 min and the prompt commands are kept.

https://github.com/continue-revolution/sd-webui-animatediff/assets/80190186/1f833d64-d3ed-48a8-9032-649161650067

ShmuelRonen commented 1 year ago

Is it possible to add tiled-upscale as part of the GUI?

continue-revolution commented 1 year ago

V2V VRAM problem should have been resolved. Let me know if problem still exists.

ShmuelRonen commented 1 year ago

Hey...

I did a comprehensive experiment regarding Tile upscale in the updated version. prompt: kid need seed to grow.

In text2image mode without Animatediff, the Tile upscale from 512X512 to 1024X1024 went quickly and successfully. See an example.

512x512 : 00001-1871256357

1024x1024: 00005-1451207930

Now I made a short gif (16 frames) in 512x512 with animatediff web-ui 00001-4036676523

When I try to resample it to 1024x1024 with tile-resample it's said the time it would take was around an hour. 1 hour

Here I gave up and did not continue.

Am I doing something wrong?

My computer have 64 GB memory and RTX3090.