Resize preprocessor cant not use another checkpoint, inpaint lama is not connect with controlnet script

[wardensc2]

Hi portu-sim. Thank for your great extension however there are some functions not working yet maybe because controlnet update or something.

I try to resize with preprocessor function and change checkpoint in my hires fix to another check or change checkpoint in your BMAB but so far it's not work. SD still use only 1 checkpoint from t2i process.

I try to use inpaint + lama and get this error

Please help us fix this, thank you in advance

[portu-sim]

Please download "control_v11p_sd15_inpaint_fp16.pt" file from internet. and put it into controlnet model to work with controlnet. If you have other inpaint model, you can change model file in "webui>setting>BMAB" Notify me the result.

[wardensc2]

I just download the control_v11p_sd15_inpaint_fp16.pt

But still have error, I have upload full error here. And this is the final error notice

And this is my extra argument in case you want to know and my machine strong enough to SD: 32gb ram, 3090

Please check the error log below

Inpaint Error.docx

PS: The BMAB still haven't allow me to change the checkpoint of Hires Step, it's force me the using the first checkpoint,

My workflow is something like this: t2i (first checkpoint) -> BMAB resized (first checkpoint) -> Hires Fix (Another checkpoint)

But BMAB ignore the second checkpoint and run hires in first checkpoint

Sorry PS 2nd time: Inpaint only is working but Inpaint+Lama is not working.

[portu-sim]

Thank you your responding. Set image width and height same like 512x512 or 768x768, it works. I don't know about that reason. Inpaint_only works well. I think that it is controlnet bug. Try inpaint_only + lama in img2img using controlnet without BMAB, same error occured.

[wardensc2]

Thank you your responding. Set image width and height same like 512x512 or 768x768, it works. I don't know about that reason. Inpaint_only works well. I think that it is controlnet bug. Try inpaint_only + lama in img2img using controlnet without BMAB, same error occured.

Can you check about your extension does not use the checkpoint in the Hires Fix if I use another checkpoint. I see in the log file it still apply loading the checkpoint but the image after render still the same as the checkpoint I use in the first t2i step

[portu-sim]

1. inpaint_only + lama :

RuntimeError: cuFFT only supports dimensions whose sizes are powers of two when computing in half precision, but got a signal size of[48, 32] This error means, your cuda (nvidia) not support this function.

After changing controlnet code "sd-webui-controlnet/annotator/lama/init.py", line 20 self.device = device.get_device_for("controlnet") to self.device = "cpu" It works very well.

2. BMAB extension does not change checkpoint in Hires.fix processing. It change checkpoint for processing only BMAB process(refiner, person, hand, face detailing). (when the resize function works with inpaint, inpaint_only, inpaint+lama, it does not change checkpoint.)

[wardensc2]

When I use BMAB resize preprocessor combine with hires fix, the log file show that SD auto switch back to first t2i checkpoint when it process hires fix and not using hires fix checkpoint:

When I turn off BMAB it, hires fix function as normal

And another thing I try to use resample, it seem that error relate to incorrect controlnet model again, I already change SD settings to another tile model I have but so far not good and when rendering I notice that BMAB did change to the model of the resample process but in the final result it switch back to first t2i checkpoint again. Here is the error log

Error log:

Middle of resample process:

Final resample process using same checkpoint as the first t2i step

And your fix with inpaint + lama work, thank you, your extension is really a big leap of control image composition in SD

[portu-sim]

Download tile model from internet. use this model control_v11f1e_sd15_tile_fp16 [3b860298]. If you have other tile model, you can change model file in "webui>setting>BMAB>ControlNet tile model" Now new code released. please update new BMAB.

Is your SD-WEBUI from githum.com/automatic1111 ???

[wardensc2]

Download tile model from internet. use this model control_v11f1e_sd15_tile_fp16 [3b860298]. If you have other tile model, you can change model file in "webui>setting>BMAB>ControlNet tile model" Now new code released. please update new BMAB.

Is your SD-WEBUI from githum.com/automatic1111 ???

I used Stable Matrix, it's the same at A1111 but has the checkpoint sharing folder which I can switch between comfy, focus, SD 1.5 and SD 1.6 freely. I'm sure it copy the same folder structure like the original A1111.

The number [3b860298] after name of controlnet model is this necessary, when I replace your original controlnet model name in BMAB setting, I deleted it and don't know how to find these number.

Ok, Im updating the extension let see it work all functions properly yet

[portu-sim]

put the file control_v11f1e_sd15_tile_fp16 into controlnet/model, then you can get hash number in web-ui controlnet.

I think that forked code is different from a1111 sd-webui. BMAB extension only tested in sd-webui from a1111. (I check a1111 code. there is no function extension call in hires.fix. never happended in a1111 code).

[wardensc2]

Hi I know why hires fix checkpoint wont work as usual after comparison resize + hires fix and resize + refiner:

I use 3 different checkpoint for easy regconize workflow

t2i (checkpoint: Darksushi), resize (Dreamshaper), Refiner (level4), Hires Fix (level4)

Resize + refiner which work correct way will be some think like this:

Darksushi -> Dreamshaper to resize by runing inpaint- > Level 4 to refine -> Back to Darksushi

Here is the log:

Resize - Refiner.docx

Resize + hires fix work in wrong way will do something like this:

Darksushi -> Level4 to resize, which is wrong model (correct is Dreamshaper) then change to Darksushi (not sure why it back maybe because it is a normal process of Hires fix if we dont use BMAB) and run inpaint by using Darksushi -> Dreamshaper to run hires fix (which is resized model not hires model) -> Back to Darksushi

Here is the log:

Resize - Hires.docx

Anyway I test Refiner it working similar to Hires Fix so if you can't fix it yet I happily use Refiner

[portu-sim]

Can you disable all extension except BMAB? Is it same? I think that It's tiled vae issue. Please turn off tiled vae and check it up.

[wardensc2]

Can you disable all extension except BMAB? Is it same? I think that It's tiled vae issue. Please turn off tiled vae and check it up.

Nope, I remove everything except controlnet and BMAB and the error still happen again. This is the error log report:

Resize - Hires Broken 2nd part.docx

[portu-sim]

Resize Intermediate IS NOT CHANGING CHECKPOINT however it use inpaint, inpaint_only, inpaint_only + lama img2img. It is only for resizing, it's not fault. Resized image repainted by REFINER or hires.fix. Changing checkpoint is unnecessary operation. It just makes performance worse.

If you use hires.fix Resize Intermediate follows txt2img checkpoint. If you use hires.fix Resize Intermediate follows webui refiner checkpoint when webui refiner set. Turning off hres.fix, Resize Intermediate follows txt2img checkpoint when BMAB checkpoint not set. Turning off hres.fix, Resize Intermediate follows BMAB checkpoint when BMAB checkpoint set. "Resize Intermediate IS NOT CHANGING CHECKPOINT" is spec.

[wardensc2]

Resize Intermediate IS NOT CHANGING CHECKPOINT however it use inpaint, inpaint_only, inpaint_only + lama img2img. It is only for resizing, it's not fault. Resized image repainted by REFINER or hires.fix. Changing checkpoint is unnecessary operation. It just makes performance worse.

If you use hires.fix Resize Intermediate follows txt2img checkpoint. If you use hires.fix Resize Intermediate follows webui refiner checkpoint when webui refiner set. Turning off hres.fix, Resize Intermediate follows txt2img checkpoint when BMAB checkpoint not set. Turning off hres.fix, Resize Intermediate follows BMAB checkpoint when BMAB checkpoint set. "Resize Intermediate IS NOT CHANGING CHECKPOINT" is spec.

There is a reason I switch checkpoint with hires fix. For example: the first t2i model is very good and render the shape of feet or or toes but it's a 2D model but I like 3D image so I will switch model to achive both result: a 3D image with good shape of feet or toes. Anyway thank for your explaination. I still try to figure how to utilize all the function of BMAB.

[portu-sim]

Now I'm adding a new code of BMAB filter. It can be possible to customize. The filters written in python.

• before hires.fix upscale
• after hires.fix upscale
• Resample
• Resize Intermediate