Closed levicki closed 10 months ago
Are you using txt2img or diffusers for image generation? If its diffusers it might be us but if it's txt2img its very likely automatic1111. Let me know either way.
@saunderez
It makes absolutely no difference which generator I use for classifier images -- as soon I click Create Model button in Dreambooth tab, folders feature_extractor
and safety_checker
are created in the models\dreambooth\MODEL_NAME\working
folder which weren't being created before.
They are both referenced in model_index.json
and safety_checker is enabled by default.
I can generate NSFW images in AUTOMATIC1111 with the same [filewords]
from input images used for classifier images just fine, they are safety checked only in Dreambooth tab and nowhere else.
EDIT: It seems that's coming from the diffusers package:
https://github.com/search?q=repo%3Ahuggingface%2Fdiffusers+requires_safety_checker&type=code
Got you. I don't think this should be happening unless the safety checker is actually specified. It seems to be fixed in dev already coz my model folders don't have it. Let me take a look.
I replicated your problem and have a workaround but need to find a proper fix.
We aren't enabling it in code so I suspect its from the default config. That config gets loaded with the model which enables it anywhere the pipeline is used that doesn't have it explicitly disabled. Rather than add a bunch of lines to disable it everywhere a pipeline could load it we can just remove it from the config completely. Ideally this should be at model creation but we can fix it in post for now.
All you have to do is either delete the safety_checker section in model_index.json located in the working_dir of the model or set it's parameters to null.
This is the section to delete, and how it appears nulled out. Yours will have the set to true and valid network names for it.
"requires_safety_checker": null, "safety_checker": [ null, null ],
Once it's been neutered you can safely delete the entire safety_checker folder.
@saunderez
Thanks for looking into it.
Yes, I figured out that workaround before creating an issue, or otherwise I wouldn't have been able to provide as much details as I did.
That said, I am looking forward to the proper fix so I don't have to edit the file for every model I create because that's not the only problem — it also wastes bandwidth and makes useless disk writes (1.13 GB model.safetensors for safety_checker is added to every created model).
On a side note, I think you should also properly disable feature_extractor
since it seems to be only used for safety_checker
.
Finally, it would be nice if someone looked at https://github.com/d8ahazard/sd_dreambooth_extension/issues/1334 and tried to reproduce, I can't train extended LoRA anymore and I'd like to know what the cause might be. The only thing different from before is update to A1111 and Dreambooth.
All I can say, even if you really need NSFW filters to opinionate morality of your code, it should at least yield a warning in a console! Or abort generation! Generating a BLACK rectangle is racist btw.
Instead of manually changing this every time, is there a simple change to the code we can make to get it to work automatically?
@saunderez Any ETA on a proper fix for this one?
I just run:
sed -i 's/requires_safety_checker": true/requires_safety_checker": null/' */working/model_index.json
sed -i 's/"stable_diffusion"/null/' */working/model_index.json
sed -i 's/"StableDiffusionSafetyChecker"/null/' */working/model_index.json
in my stable-diffusion-webui/models/dreambooth
dir.
Besides, I haven't been able to train one model ever with this tool, using it just to generate class images. In https://github.com/bmaltais/kohya_ss the safety checks are simply ignored by default and nobody cares.
@paboum I don't have sed
on Windows and even if I did (which would require installing WSL which in turn requires Hyper-V which messes up VirtualBox), I wouldn't want to have to run it for every model I create.
https://learn.microsoft.com/en-us/answers/questions/656069/sed-alternative-in-windows
As I said, I am not interested in manual workarounds. This shouldn't be difficult to fix.
This issue is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days
This issue is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days
Bump.
This issue is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 5 days
@saunderez This is coming from StableDiffusionPipeline
— it is instanatiated with default parameters and defaults are to have both safety checker and feature extractor, not to mention that the requires_safety_checker
bool is by default true
in StableDiffusionPipeline
constructor.
So yes, it's coming from you using the defaults. It affects both classification image and sample image generation during training.
Is there an existing issue for this?
What happened?
Yesterday I updated to the latest version of the extension and I tried training a LoRA on some NSFW content.
The class images generated were all black, and most of them weren't even being saved to the classifiers output folder.
I checked, and under the
model\dreambooth\MODEL_NAME\working
there was asafety_checker
folder with 1.13 GBmodel.safetensors
file andconfig.json
with commit hashcb41f3a270d63d454d385fc2e4f571c487c253c5
and name/pathCompVis/stable-diffusion-safety-checker
.Also, I noticed that
model_index.json
in theworking
folder now contains:Before in my older LoRA trainings (from the same base model) it looked like this:
Steps to reproduce the problem
Commit and libraries
Initializing Dreambooth Dreambooth revision: cf086c536b141fc522ff11f6cffc8b7b12da04b9 Successfully installed fastapi-0.94.1
[+] xformers version 0.0.20 installed. [+] torch version 2.0.1+cu118 installed. [+] torchvision version 0.15.2+cu118 installed. [+] accelerate version 0.21.0 installed. [+] diffusers version 0.19.3 installed. [+] transformers version 4.30.2 installed. [+] bitsandbytes version 0.35.4 installed.
Command Line Arguments
Console logs
Additional information
No response