[Bug]: RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

NagyArminH commented 2 weeks ago

Checklist

[X] The issue exists after disabling all extensions
[ ] The issue exists on a clean installation of webui
[ ] The issue is caused by an extension, but I believe it is caused by a bug in the webui
[ ] The issue exists in the current version of the webui
[ ] The issue has not been reported before recently
[X] The issue has been reported before but has not been fixed yet

What happened?

Tried training a dataset with Hypernetwork, but ran into the error: "RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!" when clicked on Train Hypernetwork. I only have 1 GPU, a 3060 Ti

Steps to reproduce the problem

Clicked on Train on main menu
Clicked on Train, again
Added my Hypernetwork, and changed latent sampling method to deterministic
Clicked on Train Hypernetwork

What should have happened?

Training should have started

What browsers do you use to access the UI ?

Google Chrome

Sysinfo

sysinfo-2024-06-12-10-44.json

(Deleted some file path)

Console logs

https://imgur.com/a/i2OqMni

Didn't see you cant put pics here, and I saved it as a pic instead of copying it.

Additional information

No response

dejavuwl commented 2 weeks ago

Same error for me.

genialgenteel commented 1 week ago

I only get this error when trying to use IP Adapter in the Forge ControlNet. All 3rd party extensions disabled, IP Adapter still doesn't work. (I've been having issues with other ControlNets too, but IP Adapter is the one giving me this specific error message.)

2024-06-22 14:05:34,061 - ControlNet - INFO - ControlNet Input Mode: InputMode.SIMPLE██████████████████████████████| 20/20 [00:06<00:00,  3.66it/s]
2024-06-22 14:05:34,068 - ControlNet - INFO - Using preprocessor: InsightFace+CLIP-H (IPAdapter)
2024-06-22 14:05:34,069 - ControlNet - INFO - preprocessor resolution = 512
2024-06-22 14:05:34,079 - ControlNet - INFO - Current ControlNet IPAdapterPatcher: C:\Users\USERNAME\Pictures\sd.webui\webui\models\ControlNet\ip-adapter-faceid-plusv2_sd15.bin
NeverOOM Enabled for UNet (always maximize offload)
NeverOOM Enabled for VAE (always tiled)
token_merging_ratio = 0.1
C:\Users\USERNAME\Pictures\sd.webui\system\python\lib\site-packages\insightface\utils\transform.py:68: FutureWarning: `rcond` parameter will change to the default of machine precision times ``max(M, N)`` where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass `rcond=None`, to keep using the old, explicitly pass `rcond=-1`.
  P = np.linalg.lstsq(X_homo, Y)[0].T # Affine matrix. 3 x 4
INFO: InsightFace detection resolution lowered to (384, 384).
To load target model CLIPVisionModelWithProjection
Begin to load 1 model
[Memory Management] Requested SYNC Preserved Memory (MB) =  0.0
[Memory Management] SYNC Loader Disabled for  Embedding(257, 1280)
[Memory Management] Parameters Loaded to SYNC Stream (MB) =  1204.9609375
[Memory Management] Parameters Loaded to GPU (MB) =  0.62744140625
Moving model(s) has taken 0.07 seconds
*** Error running process_before_every_sampling: C:\Users\USERNAME\Pictures\sd.webui\webui\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py
    Traceback (most recent call last):
      File "C:\Users\USERNAME\Pictures\sd.webui\webui\modules\scripts.py", line 835, in process_before_every_sampling
        script.process_before_every_sampling(p, *script_args, **kwargs)
      File "C:\Users\USERNAME\Pictures\sd.webui\system\python\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "C:\Users\USERNAME\Pictures\sd.webui\webui\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py", line 555, in process_before_every_sampling
        self.process_unit_before_every_sampling(p, unit, self.current_params[i], *args, **kwargs)
      File "C:\Users\USERNAME\Pictures\sd.webui\system\python\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "C:\Users\USERNAME\Pictures\sd.webui\webui\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py", line 501, in process_unit_before_every_sampling
        params.model.process_before_every_sampling(p, cond, mask, *args, **kwargs)
      File "C:\Users\USERNAME\Pictures\sd.webui\webui\extensions-builtin\sd_forge_ipadapter\scripts\forge_ipadapter.py", line 147, in process_before_every_sampling
        unet = opIPAdapterApply(
      File "C:\Users\USERNAME\Pictures\sd.webui\webui\extensions-builtin\sd_forge_ipadapter\lib_ipadapter\IPAdapterPlus.py", line 690, in apply_ipadapter
        clip_embed = clip_vision.encode_image(image).penultimate_hidden_states
      File "C:\Users\USERNAME\Pictures\sd.webui\webui\ldm_patched\modules\clip_vision.py", line 70, in encode_image
        outputs = self.model(pixel_values=pixel_values, output_hidden_states=True)
      File "C:\Users\USERNAME\Pictures\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "C:\Users\USERNAME\Pictures\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\USERNAME\Pictures\sd.webui\system\python\lib\site-packages\transformers\models\clip\modeling_clip.py", line 1310, in forward
        vision_outputs = self.vision_model(
      File "C:\Users\USERNAME\Pictures\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "C:\Users\USERNAME\Pictures\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\USERNAME\Pictures\sd.webui\system\python\lib\site-packages\transformers\models\clip\modeling_clip.py", line 865, in forward
        hidden_states = self.embeddings(pixel_values)
      File "C:\Users\USERNAME\Pictures\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "C:\Users\USERNAME\Pictures\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\USERNAME\Pictures\sd.webui\system\python\lib\site-packages\transformers\models\clip\modeling_clip.py", line 199, in forward
        embeddings = torch.cat([class_embeds, patch_embeds], dim=1)
    RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument tensors in method wrapper_CUDA_cat)

---
To load target model BaseModel
Begin to load 1 model
[Memory Management] Requested SYNC Preserved Memory (MB) =  0.0
[Memory Management] Parameters Loaded to SYNC Stream (MB) =  1639.406135559082
[Memory Management] Parameters Loaded to GPU (MB) =  0.0
Moving model(s) has taken 0.88 seconds
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:08<00:00,  2.42it/s]
To load target model AutoencoderKL█████████████████████████████████████████████████████████████████████████████████| 20/20 [00:05<00:00,  3.69it/s]
Begin to load 1 model
[Memory Management] Requested SYNC Preserved Memory (MB) =  0.0
[Memory Management] Parameters Loaded to SYNC Stream (MB) =  159.55708122253418
[Memory Management] Parameters Loaded to GPU (MB) =  0.0
Moving model(s) has taken 0.02 seconds
VAE tiled decode: 100%|████████████████████████████████████████████████████████████████████████████████████████████| 12/12 [00:01<00:00, 11.94it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:07<00:00,  2.84it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:07<00:00,  3.69it/s]

lllyasviel / stable-diffusion-webui-forge