portu-sim / sd-webui-bmab

Auto masking and inpainting for person, face, hand. Resizing image using detection model.
GNU Affero General Public License v3.0
307 stars 30 forks source link

bmab10月30号之后的版本一直都有bug影响controlnet预处理图区域 #15

Closed XiaoCC closed 10 months ago

XiaoCC commented 10 months ago

bmab10月30号之后的版本一直都有bug,会导致局部重绘 仅蒙版重绘的时候 controlnet的预处理图区域不对,比如我重绘一张照片的人脸,没更新bmab10月30号之前的版本时候,预处理图也是仅蒙版区域的,一旦更新了bmab10月30号之后的版本,预处理图就变成全图了,导致局部重绘仅蒙版功能异常

portu-sim commented 10 months ago

I think it's a preview problem. The latest version has fixed the issue where the preview does not appear correctly. Are you using the latest version?

XiaoCC commented 10 months ago

image image image

这是10月30日版本的插件 局部重绘仅蒙版 openpose预处理图也是局部的

XiaoCC commented 10 months ago

这是11月23的版本 image image image image

预处理图区域不一致 并且预处理图有被压缩的感觉,比例失调

XiaoCC commented 10 months ago

image image

image image image image 局部重绘脸部得到的预处理图是全图 结果压缩到脸部的512*512大小后 造成的问题就是重绘的脸部部分出现了一个全身的人

portu-sim commented 10 months ago

I can't read Chinese. Can you write it in English?

portu-sim commented 10 months ago

You should lower the "Face Denoising Strength" value to below 0.4.

portu-sim commented 10 months ago

I think this is a controlnet bug. Check if it works the same with BMAB turned off. If BMAB is turned off, it does not interfere with ControlNet operation under any circumstances.

XiaoCC commented 10 months ago

a50ee323d9f7add30f10ba424ca4ff14c4a3b287 I have updated to the latest version of the plugin and still have the size issue of the controlNet preprocessing diagram

portu-sim commented 10 months ago

What you are reporting is not an error in BMAB, but an error in ControlNet.

XiaoCC commented 10 months ago

But after I close the bmab plugin, the preprocessing diagram of controlnet is normal, I smudge the area, and only redraw the mask area 512*512 image

XiaoCC commented 10 months ago

image Enabling plugins after October 30th, the preprocessed image of controllnet becomes full body Redrawing only the masked area of the face will result in a full body portrait being redrawn

XiaoCC commented 10 months ago

Equivalent to redrawing a full body portrait of the face, resulting in abnormal visuals

portu-sim commented 10 months ago

When I checked, the controlnet was operating incorrectly even when BMAB was turned off. I'll check again tonight. thanks for letting me know.

XiaoCC commented 10 months ago

image I didn't check the bmab plugin, I just installed the version after October 30th, and I got a controlNet preprocessing exception. After uninstalling or downgrading the plugin to the October 30th version, the ControlNet pre-processing image will be normal.

XiaoCC commented 10 months ago

image fc5159fe4020a1c3f6c69079adee1d9d53ba392c
I updated the latest version of the plugin yesterday and there is still an issue with the exception of the pre-processing graph,

I think it may be due to the plugin that the preprocessed image does not support only the masked area. Even if I choose only the masked area, the preprocessed image still provides the full image, not just the openpose. All other ControlNet preprocessed images provide the full image (not just the masked area)

portu-sim commented 10 months ago
image image image image

It's working well with yesterday's updated version. Are you sure that the Inpaint Area is set to Only masked?

XiaoCC commented 10 months ago

image image image image image image

XiaoCC commented 10 months ago

The resolution of the preprocessed image is also 512 512, but the content of the preprocessed image is to compress the full body image into a 512 512 canvas, not just take the part of the face

XiaoCC commented 10 months ago

image Plugins before October 30th, or preprocessed images generated after uninstalling the plugin, only generate a portion of the facial area proportionally

portu-sim commented 10 months ago
image image

Do you see anything wrong with the settings I tested?

XiaoCC commented 10 months ago

image Found the issue. There is a conflict between the version after October 30th and the Adetailer plugin. As long as both Adetailer and BMAB versions after October 30th are installed, a ControlNet pre-processing exception will appear. Adetailer can only coexist with BMAB versions before October 30th

XiaoCC commented 10 months ago

Is it because the controllNet script calls between plugins are not coordinated properly

XiaoCC commented 10 months ago

There's also the issue that Grounding doesn't seem to support cuda12.1

portu-sim commented 10 months ago

In my case the problem doesn't seem to appear because I'm not using ADetailer. Since BMAB's Face Detailer is used, it would be a good idea to contact the ADetailer developer for this issue. (BMAB alone does not seem to cause the problem.)

Due to compatibility issues, GroudingDINO has been adjusted to operate with CPU. The version that uses CUDA is a bit slow due to the problem of not being able to support MacOS, but it can be used as a CPU version.

XiaoCC commented 10 months ago

Perhaps there was an issue with my translation description. What I meant was that using Adetailer in combination with (bmab version dated October 30th) is normal, while using Adetailer with (bmab version greater than October 30th) is abnormal. Is it possible that BMab added some features after October 30th that caused a conflict with Adetailer. Do we need the author of Adetailer to fix this conflict?

XiaoCC commented 10 months ago

Or is it possible that versions of BMAB after October 30th have included the functionality of Adetailer and no longer require the Adetailer plugin?

portu-sim commented 10 months ago

It is open source and a single developer cannot test every plugin. I will look into the issue with ADetailer, but that doesn't mean I will necessarily fix it.

Try changing the ADetailer directory name from extensions/adetailer to zadetailer so that it loads behind BMAB. There is a possibility that it will be resolved.

XiaoCC commented 10 months ago

image This method does not work

XiaoCC commented 10 months ago

I tested each version and found that (1c117f2) was the last version that was compatible with other plugins

portu-sim commented 10 months ago
image

Did you restart sd-webui after changing the directory name to zadetailer? There is no problem in operating with adetailer.

XiaoCC commented 10 months ago

image image image image image

XiaoCC commented 10 months ago

To eliminate interference, I only enabled 3 plugins

XiaoCC commented 10 months ago

image

XiaoCC commented 10 months ago

image image

XiaoCC commented 10 months ago

image image The versions of the other two plugins and the SD version have not changed, and the only variable is the bmab plugin version

portu-sim commented 10 months ago

image

Leave only 3 extension in extension directory. Not only you check other extensions ... It can make trouble. other extension just MOVE to other directory. and restart sd-webui...and test.

portu-sim commented 10 months ago

I can't rollback to 10.30 version. The code is changed by other function. I don't wanna give up new function. in other way...change directory name 'sd-webui-bmab' to 'a1sd-webui-bmab' and restart webui.

XiaoCC commented 10 months ago

(https:github.com/portu-sim/sd-webui-bmab/commit/1c117f20724fa3a456ec723fe2a6af76d1b8ba1f) was the last release on 11.7

portu-sim commented 10 months ago

I CANT USE THAT CODE. I CANT ROLLBACK TO 11.7. It changed for future work and new features.

XiaoCC commented 10 months ago

Okay then I'll use the 11.7 version first, hopefully a later version will solve it,thank you

portu-sim commented 10 months ago

I know where this problem stems from. However, inevitable changes were made to implement new features, and I have no intention of going back.

change directory name 'sd-webui-bmab' to 'a1sd-webui-bmab' and restart webui.

XiaoCC commented 10 months ago

image image It's normal after changing the directory name, it's amazing

XiaoCC commented 10 months ago

Problem solved, thank you very much😄