Converting to Diffusers ... Error

[Ai-Artsca]

Converting to Diffusers ... Traceback (most recent call last): File "/content/convertodiff.py", line 1115, in convert(args) File "/content/convertodiff.py", line 1066, in convert text_encoder, vae, unet = load_models_from_stable_diffusion_checkpoint(v2_model, args.model_to_load) File "/content/convertodiff.py", line 835, in load_models_from_stable_diffusion_checkpoint checkpoint = load_checkpoint_with_text_encoder_conversion(ckpt_path) File "/content/convertodiff.py", line 817, in load_checkpoint_with_text_encoder_conversion state_dict = checkpoint["state_dict"] KeyError: 'state_dict' rm: cannot remove '/content/stable-diffusion-custom': No such file or directory Conversion error Conversion error

[elsucht]

same for me for some models. Which types are supported? safetensor, ckpt, pruned?

[Ai-Artsca]

i was trying to use a .ckpt

[TheLastBen]

Merged models don't convert for now, I'll fix it soon, try converting it to diffusers here https://huggingface.co/spaces/diffusers/convert-sd-ckpt

[Ai-Artsca]

Merged models don't convert for now, I'll fix it soon, try converting it to diffusers here https://huggingface.co/spaces/diffusers/convert-sd-ckpt

didnt work for me, im trying to use this

https://huggingface.co/darkstorm2150/Protogen_x3.4_Official_Release/blob/main/ProtoGen_X3.4.ckpt

[AI-Casanova]

Came here hoping I would find an easier workflow, but for the moment this works. Use the Automatic1111 Dreambooth extension to unpack the diffusers AND repack them to a CKPT That CKPT is now trainable here.

(I can't get that Dreambooth to work in Colab, hence the runaround)

[SorenTruelsen]

I tried with the new HassanBlend safetensor, (round 8gb), failed the same way in Dreambooth

[Omenizer]

Seems to work now, could train Protogen (but didn't like the model. Dreamlike v2 much better)!

@TheLastBen thanks for the fix 🎁

[Omenizer]

Aaand it's not working anymore? Trained 2 models yesterday, today not a single one works - tried 5 different ones :(

[TheLastBen]

what error are you getting ?

[Omenizer]

CKPT_Path: /content/gdrive/MyDrive/SD Data/Shirayuki_General_V1-fp16.ckpt Or

CKPT_Link: text hier einfügen A CKPT direct link, huggingface CKPT link or a shared CKPT from gdrive. Converting to Diffusers ... Traceback (most recent call last): File "/content/convertodiff.py", line 1115, in convert(args) File "/content/convertodiff.py", line 1066, in convert text_encoder, vae, unet = load_models_from_stable_diffusion_checkpoint(v2_model, args.model_to_load) File "/content/convertodiff.py", line 847, in load_models_from_stable_diffusion_checkpoint info = unet.load_state_dict(converted_unet_checkpoint) File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1667, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for UNet2DConditionModel: Missing key(s) in state_dict: "up_blocks.0.upsamplers.0.conv.weight", "up_blocks.0.upsamplers.0.conv.bias", "up_blocks.1.upsamplers.0.conv.weight", "up_blocks.1.upsamplers.0.conv.bias", "up_blocks.2.upsamplers.0.conv.weight", "up_blocks.2.upsamplers.0.conv.bias". Unexpected key(s) in state_dict: "up_blocks.0.attentions.2.conv.bias", "up_blocks.0.attentions.2.conv.weight". rm: cannot remove '/content/stable-diffusion-custom': No such file or directory Conversion error Conversion error Conversion error

[TheLastBen]

Did you vhoose the correct version in "custom_model_version" ?

[Omenizer]

Pretty sure I didn't touch the setting, and I only do 1.5

[TheLastBen]

if the model is public, send me the link, I'll test it

[Omenizer]

Both versions here are failing e.g.:

https://huggingface.co/hesw23168/SD_Shirayuki_Model/tree/main

[TheLastBen]

they hold a different architecture, they are not supported by the diffusers library

[Omenizer]

Is there a way to convert them? Merging, safetensors, etc?

[TheLastBen]

They probably modified the architecture of the model to avoid people training on them, but you can try converting them from a format to another then testing if they convert to diffusers