tencent-ailab / IP-Adapter

The image prompt adapter is designed to enable a pretrained text-to-image diffusion model to generate images with image prompt.
Apache License 2.0
4.46k stars 289 forks source link

Image generation error, all green #384

Open naonao-beibei opened 2 weeks ago

naonao-beibei commented 2 weeks ago

My code is as follows, generating full green color after running. Please give me some help!! `import torch from diffusers import StableDiffusionPipeline, StableDiffusionImg2ImgPipeline, StableDiffusionInpaintPipelineLegacy, \ DDIMScheduler, AutoencoderKL from PIL import Image

from ip_adapter import IPAdapter

base_model_path = r"H:/StableDiffusionWeights/Realistic_Vision_V4.0_noVAE" vae_model_path = r"H:/StableDiffusionWeights/sd-vae-ft-mse" image_encoder_path = r"H:/StableDiffusionWeights/IPAdapter/models/image_encoder/" ip_ckpt = r"H:/StableDiffusionWeights/IPAdapter/models/ip-adapter_sd15.bin" device = "cuda"

def image_grid(imgs, rows, cols): assert len(imgs) == rows * cols

w, h = imgs[0].size
grid = Image.new('RGB', size=(cols * w, rows * h))
grid_w, grid_h = grid.size

for i, img in enumerate(imgs):
    grid.paste(img, box=(i % cols * w, i // cols * h))
return grid

noise_scheduler = DDIMScheduler( num_train_timesteps=1000, beta_start=0.00085, beta_end=0.012, beta_schedule="scaled_linear", clip_sample=False, set_alpha_to_one=False, steps_offset=1, ) vae = AutoencoderKL.from_pretrained(vae_model_path, local_files_only=True).to(dtype=torch.float16)

load SD pipeline

pipe = StableDiffusionPipeline.from_pretrained( base_model_path, torch_dtype=torch.float16, scheduler=noise_scheduler, vae=vae, feature_extractor=None, safety_checker=None, local_files_only=True )

read image prompt

image = Image.open("tv.jpg") image.resize((256, 256))

load ip-adapter

ip_model = IPAdapter(pipe, image_encoder_path, ip_ckpt, device)

only image prompt

images = ip_model.generate(pil_image=image, num_samples=1, num_inference_steps=50, seed=42) images[0].show() images[0].save('demo1.jpg')`

naonao-beibei commented 2 weeks ago

tv demo1

xiaohu2015 commented 1 week ago

do you also tested the demo in the jupyter