yapianwan commented 3 days ago

How to implement img2img? ` sigmas = self.get_sigmas_karras(steps).to(self.device) print(type(init_image))

latents

    # noise = torch.randn((size[0], 4, size[1] // 8, size[2] // 8), device="cpu", generator=torch.manual_seed(seed))
    # noise = noise.to(dtype=self.pipe.unet.dtype, device=self.device)
    # latents = noise * sigmas[0].to(noise)

    # Encode the initial image
    init_image_tensor = self.pipe.image_processor.preprocess(init_image).to(self.device)
    latents_init = self.pipe.vae.encode(init_image_tensor).latent_dist.sample(generator=torch.manual_seed(seed))
    latents_init = latents_init * sigmas[0].to(latents_init)`

I tried to modify the inference code, but it failed

ToTheBeginning commented 1 day ago

You can refer to https://huggingface.co/spaces/lllyasviel/Omost/blob/main/lib_omost/pipeline.py#L393

yapianwan commented 1 day ago

You can refer to https://huggingface.co/spaces/lllyasviel/Omost/blob/main/lib_omost/pipeline.py#L393

Thank you very much for your reply, I'm a newbie to AI, and this code has given me a better understanding of how diffusers work.

ToTheBeginning / PuLID

How to implement img2img?xl 1.1 #142

latents