Tensor Device mismatch error

[levoz92]

Hello. I am getting the following error:

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument index in method wrapper_CUDA__index_select)

This is my code:

import inspect, torch, PIL, random, os
import numpy as np

from typing import List, Optional, Union
from diffusers import StableDiffusionInpaintPipeline, StableDiffusionXLInpaintPipeline, DDIMScheduler

device = "cuda"
model_path = "weights/realisticStockPhoto_v20.safetensors"
refiner_path = "weights/sd_xl_refiner_0.9.safetensors"
img_url = "image.jpg"
mask_url = "mask.png"
prompt = "on a desert highway. Detailed. High resolution. Photorealistic. Soft light."
guidance_scale=3
num_samples = 3
height = 896
width = 1152
high_noise_frac = 0.7
negative_prompt = 'unrealistic, saturated, high contrast, big nose, painting, drawing, sketch, cartoon, anime, manga, render, CG, 3d, watermark, signature, label'
num_inference_steps = 25
output_dir = "outputs"

pipe = StableDiffusionXLInpaintPipeline.from_single_file(model_path,torch_dtype=torch.float16,num_in_channels=4, use_safetensors=True).to(device)

pipe.load_lora_weights(".", weight_name="weights/SDXL_FILM_PHOTOGRAPHY_STYLE_BetaV0.4.safetensors", adapter_name="film")
pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config)

refiner = StableDiffusionXLInpaintPipeline.from_single_file(refiner_path, text_encoder_2 = pipe.text_encoder_2, vae=pipe.vae, torch_dtype=torch.float16,num_in_channels=4, use_safetensors=True, variant="fp16")

def present_img(url):
    return PIL.Image.open(url)

image = present_img(img_url).resize((512, 512))
mask_image = present_img(mask_url).resize((512, 512))

generator = torch.Generator().manual_seed(random.randint(0,1000))

images = pipe(
    prompt=prompt,
    image=image,
    height = height,
    width = width,
    negative_prompt = negative_prompt,
    num_inference_steps = num_inference_steps,
    mask_image=mask_image,
    guidance_scale=guidance_scale,
    generator=generator,
    num_images_per_prompt=num_samples,
).images

images = refiner(
    prompt=prompt,
    image=image,
    mask_image=mask_image,
    num_inference_steps=num_inference_steps,
    denoising_start=high_noise_frac,
).images[0]

# Insert the initial image in the list so we can compare side by side
images.insert(0, image)

# Save images to the outputs directory
for i, img in enumerate(images):
    img.save(os.path.join(output_dir, f"image_{i}.png"))

# Print a message to confirm the images have been saved
print(f"All images have been saved to {output_dir}/")

Reproduction

The suggestions made online against this error is to make sure to move to gpu using .to(cuda). However, I am already doing that.

Logs

No response

System Info

Ubuntu 22.04 Huggingface Colab L4 GPU

Who can help?

No response

[weiweiwang]

@levoz92 Hi, my friend, have you ever solved this tensor device mismatch problem?

I encounter the same problem