Mikubill / sd-webui-controlnet

WebUI extension for ControlNet
GNU General Public License v3.0
16.55k stars 1.92k forks source link

[Bug]: with Deforum: TypeError: PlugableControlModel.__init__() got an unexpected keyword argument 'weight' #511

Closed Nebuluss closed 1 year ago

Nebuluss commented 1 year ago

Is there an existing issue for this?

What happened?

The model is now loading properly and accessible both in img2img and in deforum.

It's work fine in img2img but in deforum section i got an issue, it will process and work fine on the first frame then when generating the second frame an error happen :

File "D:\WebuiMirror\stable-diffusion-webui-master\modules\call_queue.py", line 56, in f res = list(func(*args, *kwargs)) File "D:\WebuiMirror\stable-diffusion-webui-master\modules\call_queue.py", line 37, in f res = func(args, *kwargs) File "D:\WebuiMirror\stable-diffusion-webui-master\extensions\deforum-for-automatic1111-webui\scripts\deforum.py", line 80, in run_deforum render_animation(args, anim_args, video_args, parseq_args, loop_args, controlnet_args, root.animation_prompts, root) File "D:\WebuiMirror\stable-diffusion-webui-master/extensions/deforum-for-automatic1111-webui/scripts\deforum_helpers\render.py", line 420, in render_animation image = generate(args, anim_args, loop_args, controlnet_args, root, frame_idx, sampler_name=scheduled_sampler_name) File "D:\WebuiMirror\stable-diffusion-webui-master/extensions/deforum-for-automatic1111-webui/scripts\deforum_helpers\generate.py", line 209, in generate processed = process_img2img_with_controlnet(p, args, anim_args, loop_args, controlnet_args, root, frame) File "D:\WebuiMirror\stable-diffusion-webui-master/extensions/deforum-for-automatic1111-webui/scripts\deforum_helpers\deforum_controlnet.py", line 341, in process_img2img_with_controlnet processed = process(p, (p.script_args)) File "D:\WebuiMirror\stable-diffusion-webui-master/extensions/deforum-for-automatic1111-webui/scripts\deforum_helpers\deforum_controlnet_hardcode.py", line 95, in process network = network_module( TypeError: PlugableControlModel.init() got an unexpected keyword argument 'weight'

Steps to reproduce the problem

  1. Go to ....
  2. Press ....
  3. ...

What should have happened?

Should generate the second frame

Commit where the problem happens

webui: 0cc0ee1 controlnet: d41c470e

Deforum 7e8fc222 (Wed Mar 1 23:28:34 2023)

What browsers do you use to access the UI ?

No response

Command Line Arguments

--xformers -api

Console logs

Launching Web UI with arguments: --xformers --api
*Deforum ControlNet support: enabled*
[AddNet] Updating model hashes...
0it [00:00, ?it/s]
[AddNet] Updating model hashes...
0it [00:00, ?it/s]
Loading weights [9aba26abdf] from D:\WebuiMirror\stable-diffusion-webui-master\models\Stable-diffusion\deliberate_v2.safetensors
Creating model from config: D:\WebuiMirror\stable-diffusion-webui-master\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Applying xformers cross attention optimization.
Textual inversion embeddings loaded(0):
Textual inversion embeddings skipped(13): cartoonish_doll, Cinema768-Digital, CinemaHelper, classipeint, dblx, kc16-v4-5000, kc32-v4-5000, mdjrny-ppc, Neg_Facelift768, protogemb, SCG768-Euphoria, SCG768-Nebula, SDA768
Model loaded in 3.7s (create model: 0.4s, apply weights to model: 0.6s, apply half(): 0.6s, move model to device: 0.9s, load textual inversion embeddings: 1.1s).
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Deforum extension for auto1111 webui, v2.2b
Unpacking ControlNet base video
Exporting Video Frames (1 every 1) frames to D:\WebuiMirror\stable-diffusion-webui-master\outputs/img2img-images\Deforum\controlnet_inputframes...
Trying to extract frames from video with input FPS of 15.0. Please wait patiently.
Successfully extracted 349 frames from video.
Loading 120 input frames from D:\WebuiMirror\stable-diffusion-webui-master\outputs/img2img-images\Deforum\controlnet_inputframes and saving video frames to D:\WebuiMirror\stable-diffusion-webui-master\outputs/img2img-images\Deforum
ControlNet base video unpacked!
Saving animation frames to:
D:\WebuiMirror\stable-diffusion-webui-master\outputs/img2img-images\Deforum
Got a request to stitch frames to video using FFmpeg.
Frames:
D:\WebuiMirror\stable-diffusion-webui-master\outputs/img2img-images\Deforum\20230306121952_%05d.png
To Video:
D:\WebuiMirror\stable-diffusion-webui-master\outputs/img2img-images\Deforum\20230306121952.mp4
Stitching *video*...
Video stitching done in 0.21 seconds!
** FFMPEG DID NOT STITCH ANY VIDEO ** Error: [Errno 2] No such file or directory: 'D:\\WebuiMirror\\stable-diffusion-webui-master\\outputs/img2img-images\\Deforum\\20230306121952.mp4'
Deforum extension for auto1111 webui, v2.2b
Unpacking ControlNet base video
Exporting Video Frames (1 every 1) frames to D:\WebuiMirror\stable-diffusion-webui-master\outputs/img2img-images\Deforum\controlnet_inputframes...
Trying to extract frames from video with input FPS of 15.0. Please wait patiently.
Successfully extracted 349 frames from video.
Loading 120 input frames from D:\WebuiMirror\stable-diffusion-webui-master\outputs/img2img-images\Deforum\controlnet_inputframes and saving video frames to D:\WebuiMirror\stable-diffusion-webui-master\outputs/img2img-images\Deforum
ControlNet base video unpacked!
Saving animation frames to:
D:\WebuiMirror\stable-diffusion-webui-master\outputs/img2img-images\Deforum
Animation frame: 0/120
Seed: 558109212
Prompt: A woman with dress walking in an airport hall, looking at screen displaying flight by Greg Rutkowski, james gurney, mignola, craig mullins, brom
Neg Prompt: nude,canvas frame, cartoon, 3d, ((disfigured)), ((bad art)), ((deformed)),((extra limbs)),((close up)),((b&w)), wierd colors, blurry, (((duplicate))), ((morbid)), ((mutilated)), [out of frame], extra fingers, mutated hands, ((poorly drawn hands)), ((poorly drawn face)), (((mutation))), (((deformed))), ((ugly)), blurry, ((bad anatomy)), (((bad proportions))), ((extra limbs)), cloned face, (((disfigured))), out of frame, ugly, extra limbs, (bad anatomy), gross proportions, (malformed limbs), ((missing arms)), ((missing legs)), (((extra arms))), (((extra legs))), mutated hands, (fused fingers), (too many fingers), (((long neck))), Photoshop, video game, ugly, tiling, poorly drawn hands, poorly drawn feet, poorly drawn face, out of frame, mutation, mutated, extra limbs, extra legs, extra arms, disfigured, deformed, cross-eye, body out of frame, blurry, bad art, bad anatomy, 3d render
╭─────┬──────┬────┬────╮
│Angle│ Zoom │Tr X│Tr Y│
├─────┼──────┼────┼────┤
│ 0.0 │1.0025│0.0 │0.0 │
╰─────┴──────┴────┴────╯
╭─────┬───┬───────────────────╮
│Steps│CFG│      Denoise      │
├─────┼───┼───────────────────┤
│ 25  │7.0│0.44999999999999996│
╰─────┴───┴───────────────────╯
Reading ControlNet base frame 0 at D:\WebuiMirror\stable-diffusion-webui-master\outputs/img2img-images\Deforum\controlnet_inputframes\00000.jpg
Reading ControlNet mask frame 0 at D:\WebuiMirror\stable-diffusion-webui-master\outputs/img2img-images\Deforum\controlnet_maskframes\00000.jpg
Neither the base nor the masking frames for ControlNet were found. Using the regular pipeline
100%|██████████████████████████████████████████████████████████████████████████████████| 12/12 [00:02<00:00,  5.14it/s]
Animation frame: 2/120  ▍                                                             | 12/540 [00:02<01:35,  5.51it/s]
 Creating in-between frame: 0; tween:0.50;
 Creating in-between frame: 1; tween:1.00;
Seed: 558109213
Prompt: A woman with dress walking in an airport hall, looking at screen displaying flight by Greg Rutkowski, james gurney, mignola, craig mullins, brom
Neg Prompt: nude,canvas frame, cartoon, 3d, ((disfigured)), ((bad art)), ((deformed)),((extra limbs)),((close up)),((b&w)), wierd colors, blurry, (((duplicate))), ((morbid)), ((mutilated)), [out of frame], extra fingers, mutated hands, ((poorly drawn hands)), ((poorly drawn face)), (((mutation))), (((deformed))), ((ugly)), blurry, ((bad anatomy)), (((bad proportions))), ((extra limbs)), cloned face, (((disfigured))), out of frame, ugly, extra limbs, (bad anatomy), gross proportions, (malformed limbs), ((missing arms)), ((missing legs)), (((extra arms))), (((extra legs))), mutated hands, (fused fingers), (too many fingers), (((long neck))), Photoshop, video game, ugly, tiling, poorly drawn hands, poorly drawn feet, poorly drawn face, out of frame, mutation, mutated, extra limbs, extra legs, extra arms, disfigured, deformed, cross-eye, body out of frame, blurry, bad art, bad anatomy, 3d render
╭─────┬────────┬────┬────╮
│Angle│  Zoom  │Tr X│Tr Y│
├─────┼────────┼────┼────┤
│ 0.0 │1.003017│0.0 │0.0 │
╰─────┴────────┴────┴────╯
╭─────┬───┬───────╮
│Steps│CFG│Denoise│
├─────┼───┼───────┤
│ 25  │7.0│ 0.35  │
╰─────┴───┴───────╯
Reading ControlNet base frame 2 at D:\WebuiMirror\stable-diffusion-webui-master\outputs/img2img-images\Deforum\controlnet_inputframes\00002.jpg
Reading ControlNet mask frame 2 at D:\WebuiMirror\stable-diffusion-webui-master\outputs/img2img-images\Deforum\controlnet_maskframes\00002.jpg
                                                ControlNet params
╭────────┬──────────────────────┬──────┬────────┬────────┬────────────────────────┬────────┬────────┬─────┬─────╮
│ module │        model         │weight│guidance│scribble│         resize         │rgb->bgr│proc res│thr a│thr b│
├────────┼──────────────────────┼──────┼────────┼────────┼────────────────────────┼────────┼────────┼─────┼─────┤
│openpose│control_sd15_openpose │  1   │   1    │ False  │Scale to Fit (Inner Fit)│ False  │  512   │ 64  │ 64  │
╰────────┴──────────────────────┴──────┴────────┴────────┴────────────────────────┴────────┴────────┴─────┴─────╯
Loading preprocessor: openpose, model: control_sd15_openpose [fef5e48e]
Loaded state_dict from [D:\WebuiMirror\stable-diffusion-webui-master\models\ControlNet\control_sd15_openpose.pth]
Deforum progress:   2%|█▍                                                             | 12/540 [00:05<03:50,  2.29it/s]
Error completing request
Arguments: ('task(hi8ibj9z2gniftj)', None, False, '', '2D', 120, 'replicate', '0:(0)', '0:(1.0025+0.002*sin(1.25*3.14*t/30))', '0:(0)', '0:(0)', '0:(1.75)', '0:(0.5)', '0:(0.5)', '0:(0)', '0:(0)', '0:(0)', False, '0:(0)', '0:(0)', '0:(0)', '0:(53)', '0: (0.065)', '0: (0.65)', '0: (1.0)', '0: (7)', '0:(1.5)', False, '0:(1)', '0:(0)', False, '0: (25)', '0: (70)', '0: (1)', '0: (200)', '0: (10000)', '0:(5), 1:(-1), 219:(-1), 220:(5)', False, '0: ("Euler a")', '0: ("!({everywhere}^({init_mask}|{video_mask}) ) ")', False, '0: ("!({everywhere}^({init_mask}|{video_mask}) ) ")', False, '0: ("model1.ckpt"), 100: ("model2.ckpt")', False, '0: (2)', '0: (5)', '0: (1.0)', '0: (0.35)', '0: (0.0)', 'Match Frame 0 LAB', 1.0, False, '2', 'perlin', 8, 8, 4, 0.5, True, 0.2, 'border', 'bicubic', False, 'D:\\WebuiMirror\\stable-diffusion-webui-master\\outputs\\img2img-images\\ControlNetAirport\\inputframes\\20230226105723.mp4', 1, 0, -1, False, False, '/content/video_in.mp4', False, '20220829210106', '{\n    "0": "A woman with dress walking in an airport hall, looking at screen displaying flight by Greg Rutkowski, james gurney, mignola, craig mullins, brom --neg nude,canvas frame, cartoon, 3d, ((disfigured)), ((bad art)), ((deformed)),((extra limbs)),((close up)),((b&w)), wierd colors, blurry, (((duplicate))), ((morbid)), ((mutilated)), [out of frame], extra fingers, mutated hands, ((poorly drawn hands)), ((poorly drawn face)), (((mutation))), (((deformed))), ((ugly)), blurry, ((bad anatomy)), (((bad proportions))), ((extra limbs)), cloned face, (((disfigured))), out of frame, ugly, extra limbs, (bad anatomy), gross proportions, (malformed limbs), ((missing arms)), ((missing legs)), (((extra arms))), (((extra legs))), mutated hands, (fused fingers), (too many fingers), (((long neck))), Photoshop, video game, ugly, tiling, poorly drawn hands, poorly drawn feet, poorly drawn face, out of frame, mutation, mutated, extra limbs, extra legs, extra arms, disfigured, deformed, cross-eye, body out of frame, blurry, bad art, bad anatomy, 3d render"\n}\n    ', '', '', 704, 384, False, False, -1, 'Euler a', False, 0, 0, 25, 0.0, 1, True, True, False, False, False, 'Deforum', '{timestring}_{index}_{prompt}.png', 'iter', 1, True, False, False, 0.55, 'https://github.com/hithereai/d/releases/download/m/kaba.png', False, False, False, True, 'https://github.com/hithereai/d/releases/download/m/mask.jpg', 1, 1, 4, 1, True, 4, 'reroll', False, 15, False, 'FFMPEG mp4', 'D:\\WebuiMirror\\stable-diffusion-webui-master\\venv\\lib\\site-packages\\imageio_ffmpeg\\binaries\\ffmpeg-win64-v4.2.2.exe', '17', 'slow', 'None', 'https://freetestdata.com/wp-content/uploads/2021/09/Free_Test_Data_1MB_MP3.mp3', False, 'realesr-animevideov3', 'x2', True, False, 'x0_pred', 'C:/SD/20230124234916_%05d.png', 'testvidmanualsettings.mp4', False, 'None', 2, False, 2, False, '', True, False, 'None', True, 'None', False, 'Farneback', False, 'None', False, 'None', False, False, '0:(1)', '0:(0.5)', '0:(1)', '0:(100)', '0:(0)', False, '{\n    "0": "https://user-images.githubusercontent.com/121192995/215279228-1673df8a-f919-4380-b04c-19379b2041ff.png",\n    "50": "https://user-images.githubusercontent.com/121192995/215279281-7989fd6f-4b9b-4d90-9887-b7960edd59f8.png",\n    "100": "https://user-images.githubusercontent.com/121192995/215279284-afc14543-d220-4142-bbf4-503776ca2b8b.png",\n    "150": "https://user-images.githubusercontent.com/121192995/215279286-23378635-85b3-4457-b248-23e62c048049.jpg",\n    "200": "https://user-images.githubusercontent.com/121192995/215279228-1673df8a-f919-4380-b04c-19379b2041ff.png"\n}', '0:(0.75)', '0:(0.35)', '0:(0.25)', '0:(20)', '0:(0.075)', <tempfile._TemporaryFileWrapper object at 0x000002C8332CE290>, None, True, False, False, False, 'openpose', 'control_sd15_openpose [fef5e48e]', 1, 1, 512, 64, 64, 'Scale to Fit (Inner Fit)') {}
Traceback (most recent call last):
  File "D:\WebuiMirror\stable-diffusion-webui-master\modules\call_queue.py", line 56, in f
    res = list(func(*args, **kwargs))
  File "D:\WebuiMirror\stable-diffusion-webui-master\modules\call_queue.py", line 37, in f
    res = func(*args, **kwargs)
  File "D:\WebuiMirror\stable-diffusion-webui-master\extensions\deforum-for-automatic1111-webui\scripts\deforum.py", line 80, in run_deforum
    render_animation(args, anim_args, video_args, parseq_args, loop_args, controlnet_args, root.animation_prompts, root)
  File "D:\WebuiMirror\stable-diffusion-webui-master/extensions/deforum-for-automatic1111-webui/scripts\deforum_helpers\render.py", line 420, in render_animation
    image = generate(args, anim_args, loop_args, controlnet_args, root, frame_idx, sampler_name=scheduled_sampler_name)
  File "D:\WebuiMirror\stable-diffusion-webui-master/extensions/deforum-for-automatic1111-webui/scripts\deforum_helpers\generate.py", line 209, in generate
    processed = process_img2img_with_controlnet(p, args, anim_args, loop_args, controlnet_args, root, frame)
  File "D:\WebuiMirror\stable-diffusion-webui-master/extensions/deforum-for-automatic1111-webui/scripts\deforum_helpers\deforum_controlnet.py", line 341, in process_img2img_with_controlnet
    processed = process(p, *(p.script_args))
  File "D:\WebuiMirror\stable-diffusion-webui-master/extensions/deforum-for-automatic1111-webui/scripts\deforum_helpers\deforum_controlnet_hardcode.py", line 95, in process
    network = network_module(
TypeError: PlugableControlModel.__init__() got an unexpected keyword argument 'weight'

Additional information

none

numz commented 1 year ago

extacly same issue here

Mikubill commented 1 year ago

Since we have changed the input args in PlugableControlModel, deforum_controlnet_hardcode needs to be update to match it.

For developers, We recommend to use external extension API - see https://github.com/Mikubill/sd-webui-controlnet/wiki/API for details.

hithereai commented 1 year ago

Fixed long time ago. Issue can be closed.

Adamanrt commented 1 year ago

блять кто нибудь помогите