search

tencent-ailab / IP-Adapter

The image prompt adapter is designed to enable a pretrained text-to-image diffusion model to generate images with image prompt.
Apache License 2.0
5.11k stars 333 forks source link

Testing Issues #164

Open 786039663 opened 10 months ago

786039663 commented 10 months ago

After I converted the model.safetensors file I got after training to ip_adapter.bin, when I tested it, it reported an error. image image

xiaohu2015 commented 10 months ago

you use the code in https://github.com/tencent-ailab/IP-Adapter#how-to-train?

786039663 commented 10 months ago

Yes, I trained according to this https://github.com/tencent-ailab/IP-Adapter#how-to-train. After training, get model.safetensors. Then convert model.safetensors to ip_adapter.bin Finally an error when loading ip_adapter.bin image image image image image

xiaohu2015 commented 10 months ago

you can load the ip_adapter.bin, and print the keys:

import torch

sd = torch.load("ip_adapter.bin")

print(sd["image_proj"].keys())
print(sd["ip_adapter"].keys())
786039663 commented 10 months ago

That's the result, but I don't know what it means. image

786039663 commented 10 months ago

The training was prompted with this message. image

xiaohu2015 commented 10 months ago

It seems that some weights are missing, but I don't know where the problem lies. you can try this:

import torch
ckpt = "checkpoint-50000/pytorch_model.bin"

sd = torch.load(ckpt, map_location="cpu")
image_proj_sd = {}
ip_sd = {}
for k in sd:
    if k.startswith("image_proj_model"):
        image_proj_sd[k.replace("image_proj_model.", "")] = sd[k]
    elif "_ip." in k: 
        ip_sd[k.replace("unet.", "")] = sd[k] 

torch.save({"image_proj": image_proj_sd, "ip_adapter": ip_sd}, "ip_adapter.bin")
masaisai111 commented 6 months ago

Have you solved the problem

系统提示培训并显示此消息。 图像