search

tencent-ailab / IP-Adapter

The image prompt adapter is designed to enable a pretrained text-to-image diffusion model to generate images with image prompt.
Apache License 2.0
5.33k stars 337 forks source link

KeyError: 'Adapter' #444

Open FunkyFrog1 opened 1 week ago

FunkyFrog1 commented 1 week ago

Error while loading adapter It seems that i used wrong ckpt, but I've try './IP-Adapter/sdxl_models/ip-adapter_sdxl.bin' './IP-Adapter/sdxl_models/ip-adapter_sdxl.safetensors' './IP-Adapter/sdxl_models/ip-adapter_sdxl_vit-h.bin' './IP-Adapter/sdxl_models/ip-adapter_sdxl_vit-h.safetensors' And It didn't work

Loading pipeline components...: 100%|██████████| 7/7 [00:00<00:00, 8.14it/s] Traceback (most recent call last): File "/mnt/e37aa9fa-bd22-4694-ac4e-d5ecb9d58c1c/brain/SDXL/SDXL_Adapter.py", line 43, in main() File "/mnt/e37aa9fa-bd22-4694-ac4e-d5ecb9d58c1c/brain/SDXL/SDXL_Adapter.py", line 29, in main ip_model = IPAdapterXL(pipe, image_encoder_path, ip_ckpt, 'cuda') File "/mnt/e37aa9fa-bd22-4694-ac4e-d5ecb9d58c1c/brain/SDXL/Adapter/ip_adapter.py", line 84, in init self.load_ip_adapter() File "/mnt/e37aa9fa-bd22-4694-ac4e-d5ecb9d58c1c/brain/SDXL/Adapter/ip_adapter.py", line 137, in load_ip_adapter ip_layers.load_state_dict(state_dict["Adapter"]) KeyError: 'Adapter'

And here is my code

pipe = load_pipe()
image_encoder_path = './IP-Adapter/sdxl_models/image_encoder'
ip_ckpt = './IP-Adapter/sdxl_models/ip-adapter_sdxl.bin'
ip_model = IPAdapterXL(pipe, image_encoder_path, ip_ckpt, 'cuda')

I refer to the demo

And I download the whole files from here

My packages are: diffusers==0.28.0 torch==2.3.0 tokenizers==0.19.1 transformers==4.41.1

FunkyFrog1 commented 1 week ago 
def load_ip_adapter(self):
    if os.path.splitext(self.ip_ckpt)[-1] == ".safetensors":
        state_dict = {"image_proj": {}, "Adapter": {}}
        with safe_open(self.ip_ckpt, framework="pt", device="cpu") as f:
            for key in f.keys():
                print(key)
                if key.startswith("image_proj."):
                    state_dict["image_proj"][key.replace("image_proj.", "")] = f.get_tensor(key)
                elif key.startswith("ip_adapter."):
                    state_dict["Adapter"][key.replace("ip_adapter.", "")] = f.get_tensor(key)
    else:
        state_dict = torch.load(self.ip_ckpt, map_location="cpu")
    self.image_proj_model.load_state_dict(state_dict["image_proj"])
    ip_layers = torch.nn.ModuleList(self.pipe.unet.attn_processors.values())
    ip_layers.load_state_dict(state_dict["Adapter"])

Use safetensors format and replace load_ip_adapter() function in Class IPAdapter may works