zappityzap
commented
5 months ago Thanks for the detailed report. If I'm reading the controlnet code correctly, there should be additional lines of log output right after "batch control images:" that say "batch control keyframe index:". So something is going wrong with the keyframe parameter. I'll see if I can reproduce this, but if you want to try some coding... I would add logger.info lines like this:
logger.info(f"\tbatch control images: {batch_image_dir}")
logger.info(f"\tbatch parameters: {batch_parameters}")
for ad_cn_batch_parameter in batch_parameters[1:]:
logger.info(f"\t\tad_cn_batch_parameter: {ad_cn_batch_parameter}")
if ad_cn_batch_parameter.startswith("mask:"):
NergiZed
Is there an existing issue for this?
Have you read FAQ on README?
What happened?
Trying to use SpareCtrl, but it seems to just fail and generates the indicated number of frames with animateDiff without the influence of SparseCtrl.
I read the documents as best as I could, read issues relating to SparseCtrl (there are only 4), and gathered what I believed to be the right files. AnimateDiff with Multi-ControelNets works fine, so it's just the SparseCtrl part. Here's the basics:
webui: Automatic1111 version 1.9.3
extensions: sd-webui-animatediff sd-webui-controlnet
files: The motion module and sparsectrl files I got from here: https://huggingface.co/conrevo/AnimateDiff-A1111/tree/main/control sparsectrl: file: mm_sd15_v3_sparsectrl_rgb.safetensors location: C:\AI\SD\stable-diffusion-webui-xl\models\ControlNet size: 947 MB (993,974,272 bytes) motion module: file: mm_sd15_v3 location: C:\AI\SD\stable-diffusion-webui-xl\extensions\sd-webui-animatediff\model size: 797 MB (836,618,008 bytes)
Parameters prompt and neg:
generation settings:
AnimateDiff settings:
ControlNet settings:
Spherically, for the input directory: D:_work\slipperygem\hentai\mouse in the house\scenes\tests\color diffuse test\SparseControlTest keyframe:0,6,12,18,24
The error I get has to do with the keyframe part of that. Here is the error
That's pointing to this batch.py file here: C:\AI\SD\stable-diffusion-webui-xl\extensions\sd-webui-controlnet\scripts\animate_diff Specifically this chunk:
for ad_cn_batch_parameter in batch_parameters[1:]: if ad_cn_batch_parameter.startswith("mask:"): unit.batch_mask_dir = ad_cn_batch_parameter[len("mask:") :].strip() logger.info(f"\tbatch control mask: {unit.batch_mask_dir}") elif ad_cn_batch_parameter.startswith("keyframe:"): unit.batch_keyframe_idx = ad_cn_batch_parameter[len("keyframe:") :].strip() unit.batch_keyframe_idx = [ int(b_i.strip()) for b_i in unit.batch_keyframe_idx.split(",") ] logger.info(f"\tbatch control keyframe index: {unit.batch_keyframe_idx}")Its looking for the text 'keyframe:' in the 'batch_parameters', so I'm not sure what I'm doing wrong. I'm copying the format from the documentation:
I'm using these 5 keyframes for the test:
And the output is, unsurprisingly, not influenced by the keyframes at all. Clearly it just skipped it and gave a pure prompt AnimateDiff output:
The goal of the test was to use SpareCtrl to recreate the original gen as much as possible, which is a multi-ControlNet guided generation with animDiff.
I'm not sure what other information I can provide. Seems like it could be minor typo type thing, but I just don't know what's wrong. Know that I can not code well, and because of that I almost never write issues; I don't think I can do a good job of it. I would much prefer to either figure it out or wait it out. However, SpareCtrl is a feature that I desperately want to learn how to use, and there's just not a whole lot of information out there about it. Please help.
Anyhow, thanks for your time, apologies if it was some braindead issue on my part.
Great extension by the way. just got back into it, and the potential is incredible.
Steps to reproduce the problem
What should have happened?
It should interpolate the frames provided according to the assigned keyframes over the indicated frame length. I think.
Commit where the problem happens
webui: https://github.com/AUTOMATIC1111/stable-diffusion-webui/commit/1c0a0c4c26f78c32095ebc7f8af82f5c04fca8c0 sd-webui-animatediff: https://github.com/continue-revolution/sd-webui-animatediff/commit/67e6b96aacd5fb5d690f4f245bb34c26b8de64e9 sd-webui-controlnet: https://github.com/Mikubill/sd-webui-controlnet/commit/e338a861dc4b4087db001c7fee8e1b0f916e0c3b
What browsers do you use to access the UI ?
No response
Command Line Arguments
Console logs
Additional information
I dunno. I guess I'm using LCM sampler as well. That works fine though. I'm using Opera to access the UI My G-card is a 4090 I'm using Windows 10 I live on the northern hemisphere I breath a mainly nitrogen oxygen atmosphere