Issue with Image Reconstruction Using LCM Scheduler

[hansam95]

Thank you for sharing your code

I am currently facing an issue when trying to reconstruct images using the LCM scheduler. Below is the code that I'm working with:

from PIL import Image
from src.eunms import Model_Type, Scheduler_Type
from src.utils.enums_utils import get_pipes
from src.config import RunConfig
from main import run as invert

model_type = Model_Type.LCM_SDXL
scheduler_type = Scheduler_Type.LCM

device = 'cuda'
pipe_inversion, pipe_inference = get_pipes(model_type, scheduler_type, device=device)

input_image = Image.open("example_images/lion.jpeg").convert("RGB").resize((512, 512))
prompt = "a lion in the field"

config = RunConfig(model_type = model_type,
                    scheduler_type = scheduler_type)

rec_img, inv_latent, noise, all_latents = invert(input_image,
                                                 prompt,
                                                 config,
                                                 pipe_inversion=pipe_inversion,
                                                 pipe_inference=pipe_inference,
                                                 do_reconstruction=True)

When using the same code with model_type set to SDXL_Turbo and scheduler_type to EULER, everything works fine. However, when using LCM, I encounter an error stating that MyCLMScheduler has no attribute step_and_update_noise. Could you please advise on how to address this issue?

Additionally, I have some questions regarding the inversion process with LCM:

The inversion process with LCM isn't discussed in the paper. Specifically, lcm_scheduler.py lines 185-188 detail the main process, but I'm unclear on where the equations used are derived from. Could you provide some clarity or point me towards relevant resources that explain this?

Thank you for your help and I look forward to your response.

[ajrheng]

Hi, have you found a resolution to the problem of no step_and_update_noise? I also ran into the same issue with LCMs.