Readapting the Diffusers pipeline to the img2img setting

[sunhaozhepy]

I've seen the re-implementation of your project in Diffusers, good work! However, I'm interested in knowing whether your pipeline could be applied in the img2img setting (by this I mean I want to preserve some detail in the original image, like the img2img+ControlNet pipeline that Diffusers provide), and it seems that the current pipeline is limited to a txt2img-like setting.

Do you have any idea how this could be done? Thank you very much in advance!

[williamyang1991]

This pipeline is a img2img+ControlNet pipeline. We have a parameter strength for img2img control.

output_frames = pipe(
    "a beautiful woman in CG style, best quality, extremely detailed",
    frames,
    control_frames,
    num_inference_steps=20,
    strength=0.75,
    controlnet_conditioning_scale=0.7,
    generator=generator,
    warp_start=0.0,
    warp_end=0.1,
    mask_start=0.5,
    mask_end=0.8,
    mask_strength=0.5,
    negative_prompt='longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality'
).frames

You can find the document here

[sunhaozhepy]

Ah OK I misunderstood, thanks!