[Test] Stylize Video

[s9roll7]

1. Tile -> upscale motion_module : mm_sd_v14.ckpt steps : 20 guidance_scale : 10 [0] ip_adapter_plus ("is_plus_face": false, "is_plus": true) / scale : 0.5 controlnet_tile / controlnet_conditioning_scale : 0.75 size : 512x512 context : 16 [1] ip_adapter_plus / scale : 0.5 controlnet_tile / controlnet_conditioning_scale : 1.0 size : 1024x1024 context : 8

   
   

2. Lineart -> upscale motion_module : mm_sd_v15_v2.ckpt steps : 20 guidance_scale : 10 [0] ip_adapter_plus / scale : 0.5 controlnet_lineart / controlnet_conditioning_scale : 1.0 controlnet_ip2p / controlnet_conditioning_scale : 0.5 size : 512x512 context : 16 [1] ip_adapter_plus / scale : 0.5 controlnet_tile / controlnet_conditioning_scale : 1.0 controlnet_ip2p / controlnet_conditioning_scale : 0.5 size : 768x768 context : 8

   
   

3. Openpose -> upscale motion_module : mm_sd_v15_v2.ckpt steps : 20 guidance_scale : 10 [0] ip_adapter_plus / scale : 0.5 controlnet_openpose / controlnet_conditioning_scale : 1.0 size : 512x512 context : 16 [1] ip_adapter_plus / scale : 0.5 controlnet_tile / controlnet_conditioning_scale : 1.0 size : 1024x1024 context : 8

4. Softedge -> upscale motion_module : mm-Stabilized_high.pth steps : 20 guidance_scale : 10 [0] ip_adapter_plus / scale : 0.5 controlnet_softedge / controlnet_conditioning_scale : 1.0 controlnet_ip2p / controlnet_conditioning_scale : 0.5 size : 512x760 context : 16 [1] ip_adapter_plus / scale : 0.5 controlnet_tile / controlnet_conditioning_scale : 1.0 size : 768x1136 context : 8

[sdqq1234]

Why my result is bad when I use openpose, I updated to the latest on the morning of September 13th I used command animatediff generate -c config/prompts/Meina.json -W 424 -H 768 -L 16 -C 16 my openpose images my result MyPrompts MyPrompts.zip

[s9roll7]

MyPrompts

You are trying to apply controlnet to all frames. In that case, you have to cut the effect of controlnet on the neighboring frames

16

https://github.com/s9roll7/animatediff-cli-prompt-travel/issues/6#issuecomment-1694404949

    "controlnet_openpose":{
      "enable": true,
      "use_preprocessor":true,
      "guess_mode":false,
      "controlnet_conditioning_scale": 1.0,
      "control_guidance_start": 0.0,
      "control_guidance_end": 1.0,
      "control_scale_list":[0.5,0.4,0.3,0.2,0.1]      # <---- ng
    },

    "controlnet_openpose":{
      "enable": true,
      "use_preprocessor":true,
      "guess_mode":false,
      "controlnet_conditioning_scale": 1.0,
      "control_guidance_start": 0.0,
      "control_guidance_end": 1.0,
      "control_scale_list":[]      # <---- ok
    },

[sdqq1234]

thank you very much, it's much better!!!

[yuturiy]

@s9roll7 can you please provide the full prompt.json for this video

I try to reproduce but fail every time

[s9roll7]

Unfortunately, it looks like I overwrote the config file and it didn't remain. What exactly happens when you say "fail"? The video file is not generated?

[yuturiy]

Unfortunately, it looks like I overwrote the config file and it didn't remain. What exactly happens when you say "fail"? The video file is not generated?

Sorry. By fail I mean I get very inconsistent video. So it would be nice to have a complete prompt example where everything is consistent.

[s9roll7]

Are you using lora? it is quite difficult to keep the character and style consistent without lora.

[yuturiy]

Are you using lora? it is quite difficult to keep the character and style consistent without lora.

no, there was nothing about loras near stylize examples( will try, thanks

[tokimwc]

Hello @s9roll7

I've been exploring your repository and noticed the mention of VAE in the configurations. However, I couldn't find any specific guidance on where to place the VAE within the repository. Could you provide some insights or directions on this?

Thank you for your assistance!

[s9roll7]

Currently, it is not possible to read vae by itself. The data that the checkpoint has is used.

[seemon2]

Hi, .. when running the command in the command "animatediff stylize create-config IA.mp4" (my video), encounter this error ..

Is there an issue that need resolve that I am not aware of for stylized run?

[rudekuclux]

Why my result is bad when I use openpose, I updated to the latest on the morning of September 13th I used command animatediff generate -c config/prompts/Meina.json -W 424 -H 768 -L 16 -C 16 my openpose images my result MyPrompts MyPrompts.zip

i have issue like this how do you fix it?

[s9roll7]

Why my result is bad when I use openpose, I updated to the latest on the morning of September 13th I used command animatediff generate -c config/prompts/Meina.json -W 424 -H 768 -L 16 -C 16 my openpose images my result MyPrompts MyPrompts.zip

i have issue like this how do you fix it?

https://github.com/s9roll7/animatediff-cli-prompt-travel/issues/29#issuecomment-1717394590

[prezofpizza]

Can anyone share some prompts that have had good results? My results tend to look pretty messy.

[lyc6749]

4. controlnet_conditioning_scale

Have you solved this problem?