IP-Adapter does not work in controlnet (Resolved, it works)

SunGreen777 commented 8 months ago

Is there an existing issue for this?

[X] I have searched the existing issues and checked the recent builds/commits

What happened?

IP-Adapter does not work in controlnet, There are no errors, the effect is as if controlnet is not enabled. Other models work.

Steps to reproduce the problem

What should have happened?

Sysinfo

@echo off set PYTHON= set GIT= set VENV_DIR= set GIT_SSL_NO_VERIFY=true git pull set COMMANDLINE_ARGS= --opt-sub-quad-attention --medvram --api --no-half --disable-nan-check set SAFETENSORS_FAST_GPU=1 call webui.bat

What browsers do you use to access the UI ?

Google Chrome

Console logs

2023-11-02 00:52:17,246 - ControlNet - INFO - Loading model: ip-adapter-plus_sd15 [c817b455]
2023-11-02 00:52:17,293 - ControlNet - INFO - Loaded state_dict from [W:\stable-diffusion-webui-directml\models\ControlNet\ip-adapter-plus_sd15.pth]
2023-11-02 00:52:17,542 - ControlNet - INFO - ControlNet model ip-adapter-plus_sd15 [c817b455] loaded.
2023-11-02 00:52:17,549 - ControlNet - INFO - Loading preprocessor: ip-adapter_clip_sd15
2023-11-02 00:52:17,550 - ControlNet - INFO - preprocessor resolution = 512
*** Error running process: W:\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "W:\stable-diffusion-webui-directml\modules\scripts.py", line 619, in process
        script.process(p, *script_args)
      File "W:\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\controlnet.py", line 977, in process
        self.controlnet_hack(p)
      File "W:\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\controlnet.py", line 966, in controlnet_hack
        self.controlnet_main_entry(p)
      File "W:\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\controlnet.py", line 808, in controlnet_main_entry
        detected_map, is_image = preprocessor(
      File "W:\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\utils.py", line 75, in decorated_func
        return cached_func(*args, **kwargs)
      File "W:\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\utils.py", line 63, in cached_func
        return func(*args, **kwargs)
      File "W:\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\global_state.py", line 35, in unified_preprocessor
        return preprocessor_modules[preprocessor_name](*args, **kwargs)
      File "W:\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\scripts\processor.py", line 350, in clip
        from annotator.clipvision import ClipVisionDetector
      File "W:\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\clipvision\__init__.py", line 82, in <module>
        clip_vision_h_uc = torch.load(clip_vision_h_uc)['uc']
      File "W:\stable-diffusion-webui-directml\modules\safe.py", line 108, in load
        return load_with_extra(filename, *args, extra_handler=global_extra_handler, **kwargs)
      File "W:\stable-diffusion-webui-directml\modules\safe.py", line 156, in load_with_extra
        return unsafe_torch_load(filename, *args, **kwargs)
      File "W:\stable-diffusion-webui-directml\venv\lib\site-packages\torch\serialization.py", line 809, in load
        return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
      File "W:\stable-diffusion-webui-directml\venv\lib\site-packages\torch\serialization.py", line 1172, in _load
        result = unpickler.load()
      File "C:\Users\Havemoney\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 1213, in load
        dispatch[key[0]](self)
      File "C:\Users\Havemoney\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 1254, in load_binpersid
        self.append(self.persistent_load(pid))
      File "W:\stable-diffusion-webui-directml\venv\lib\site-packages\torch\serialization.py", line 1142, in persistent_load
        typed_storage = load_tensor(dtype, nbytes, key, _maybe_decode_ascii(location))
      File "W:\stable-diffusion-webui-directml\venv\lib\site-packages\torch\serialization.py", line 1116, in load_tensor
        wrap_storage=restore_location(storage, location),
      File "W:\stable-diffusion-webui-directml\venv\lib\site-packages\torch\serialization.py", line 217, in default_restore_location
        result = fn(storage, location)
      File "W:\stable-diffusion-webui-directml\venv\lib\site-packages\torch\serialization.py", line 182, in _cuda_deserialize
        device = validate_cuda_device(location)
      File "W:\stable-diffusion-webui-directml\venv\lib\site-packages\torch\serialization.py", line 166, in validate_cuda_device
        raise RuntimeError('Attempting to deserialize object on a CUDA '
    RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.

---
100%|█████████████████████████████████████████████████████████████████████████████████| 22/22 [00:50<00:00,  2.28s/it]
Total progress: 100%|█████████████████████████████████████████████████████████████████| 22/22 [00:48<00:00,  2.19s/it]
Total progress: 100%|█████████████████████████████████████████████████████████████████| 22/22 [00:48<00:00,  2.22s/it]

Additional information

Win 11 - update 32ram rx 570 8gb Ryzen 5 3600

oOJonnyOo commented 8 months ago

https://github.com/tencent-ailab/IP-Adapter/issues/53#issuecomment-1729322454 prolly this will help

SunGreen777 commented 8 months ago

tencent-ailab/IP-Adapter#53 (comment) prolly this will help

Not immediately but now everything works! 1 Reinstalled SD, 2 Performed a string replacement after installing CN, before rebooting the interface. 3 Apply, reboot Thanks

\stable-diffusion-webui-directml\extensions\sd-webui-controlnet\annotator\clipvision\ init.py Line 82 clip_vision_h_uc = torch.load(clip_vision_h_uc, map_location=torch.device('cpu'))['uc']

lshqqytiger / stable-diffusion-webui-amdgpu