Stable Diffusion Model Inference model weight after Training complete

[JustinKai0527]

Hi, I am currently working on reproduce your code, I got some question. the train-scripts/random_label.py the model is save by torch.save but in eval-scripts/generate_image.py I want to test my training model weight, I can't load the model weight to the unet How to solve this, thx!!! The random_label.py saving model weight code ](url) The generate_image.py loading the model weight

[a-F1]

Thank you for your interest and attention to our work! Could you please provide the specific command you used, so we can help further?

[JustinKai0527]

Thank you for responding to my question. I am trying to reproduce the imagenette unlearn on specific concept, I do the same architecture in your SD README.md, 1.clone the original stable diffusion 2. downlaod your file on SD folder paste to the original stable diffusion 3. download checkpoint. Then I do following command Based on this command I will get a mask checkpoint for specific concept, and after random_label.py train will get a model weight. My question is during inference, I want to use the random_label.py model weight to inference but there is a bug..

The command is as follows: Training Stage: python train-scripts/generate_mask.py --ckpt_path models/ldm/stable-diffusion-v1/sd-v1-4-full-ema.ckpt --class 0 --device 0 python train-scripts/random_label.py --ckpt_path models/ldm/stable-diffusion-v1/sd-v1-4-full-ema.ckpt --mask_path mask/3/with_0.5.pt --device 0 --class_to_forget 3 --train_method full

Inference Stage: python eval-scripts/generate-images.py --model_name models/ldm/stable-diffusion-v1/sd-v1-4-full-ema.ckpt --prompts_path prompts/imagenette.csv --save_path evaluation_folder I check the pretrain model can generate well then use the random_label.py get model weight python eval-scripts/generate-images.py --model_name models/3/3.pt --prompts_path prompts/imagenette.csv --save_path evaluation_folder I can't get let the model weight load from the model_name error like this Model path is not valid, please check the file name and structure: Error(s) in loading state_dict for UNet2DConditionModel: Missing key(s) in state_dict: "conv_in.weight", "conv_in.bias", "time_embedding.linear_1.weight", "time_embedding.linear_1.bias", "time_embedding.linear_2.weight", "time_embedding.linear_2.bias", "down_blocks.0.attentions.0.norm.weight", "down_blocks.0.attentions.0.norm.bias", "down_blocks.0.attentions.0.proj_in.weight", "down_blocks.0.attentions.0.proj_in.bias", "down_blocks.0.attentions.0.transformer_blocks.0.norm1.weight", "down_blocks.0.attentions.0.transformer_blocks.0.norm1.bias", "down_blocks.0.attentions.0.transformer_blocks.0.attn1.to_q.weight"

[a-F1]

Thank you for sharing your command with me. It seems you've modified the folder name for saving the model and the model name. Please save using the default folder name, which is as follows: compvis-cl-class_{str(class_toforget)}-method{trainmethod}-alpha{alpha}-epoch{epochs}-lr{lr}. Within this folder, weights of diffuser format will be saved as： diffusers-cl-class_{str(class_toforget)}-method{trainmethod}-alpha{alpha}-epoch{epochs}-lr{lr}.pt

During inference, there's no need to specify the model name ending with .pt in the 'model_name'. Simply input the folder name where the model weights are saved.