Help, high Ram usage in colab, then session crashes!

[Pranay144]

import diffusers
from diffusers.utils import load_image
from diffusers.models import ControlNetModel

import cv2
import torch
import numpy as np
from PIL import Image

from insightface.app import FaceAnalysis
from pipeline_stable_diffusion_xl_instantid_full import StableDiffusionXLInstantIDPipeline, draw_kps

# prepare 'antelopev2' under ./models
app = FaceAnalysis(name='antelopev2', root='./', providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
app.prepare(ctx_id=0, det_size=(640, 640))

# prepare models under ./checkpoints
face_adapter = f'./checkpoints/ip-adapter.bin'
controlnet_path = f'./checkpoints/ControlNetModel'

# load IdentityNet
controlnet = ControlNetModel.from_pretrained(controlnet_path, torch_dtype=torch.float16)

base_model = 'https://huggingface.co/RunDiffusion/Juggernaut-XL-v9/blob/main/Juggernaut-XL_v9_RunDiffusionPhoto_v2.safetensors' 

pipe = StableDiffusionXLInstantIDPipeline.from_single_file(
    base_model,
    controlnet=controlnet,
    torch_dtype=torch.float16
)

# crash after downloading the model,
pipe.cuda()

# load adapter
pipe.load_ip_adapter_instantid(face_adapter)

After the model download, pipe = StableDiffusionXLInstantIDPipeline.from_single_file( base_model, controlnet=controlnet, torch_dtype=torch.float16 ) The ram usage keeps going up, and then session is crashed everytime.

[haofanwang]

Refer to here. You can either use pipe.enable_vae_tiling() to reduce memory, or use distributed inference if you have more than one GPU.