PRIS-CV / DemoFusion

Let us democratise high-resolution generation! (CVPR 2024)
https://ruoyidu.github.io/demofusion/demofusion.html
1.98k stars 229 forks source link

How to load local .safetensor? #13

Closed green-anger closed 11 months ago

green-anger commented 11 months ago

DemoFusionSDXLPipeline inherits FromSingleFileMixin, so we can use pipe = DemoFusionSDXLPipeline.from_single_file(...), however inside it has:

pipeline_name = cls.__name__
...
if pipeline_name in [
    "StableDiffusionControlNetPipeline",
    "StableDiffusionControlNetImg2ImgPipeline",
    "StableDiffusionControlNetInpaintPipeline",
]:
    from .models.controlnet import ControlNetModel
    from .pipelines.controlnet.multicontrolnet import MultiControlNetModel

    # Model type will be inferred from the checkpoint.
    if not isinstance(controlnet, (ControlNetModel, MultiControlNetModel)):
        raise ValueError("ControlNet needs to be passed if loading from ControlNet pipeline.")
elif "StableDiffusion" in pipeline_name:
    # Model type will be inferred from the checkpoint.
    pass
elif pipeline_name == "StableUnCLIPPipeline":
    model_type = "FrozenOpenCLIPEmbedder"
    stable_unclip = "txt2img"
elif pipeline_name == "StableUnCLIPImg2ImgPipeline":
    model_type = "FrozenOpenCLIPEmbedder"
    stable_unclip = "img2img"
elif pipeline_name == "PaintByExamplePipeline":
    model_type = "PaintByExample"
elif pipeline_name == "LDMTextToImagePipeline":
    model_type = "LDMTextToImage"
else:
    raise ValueError(f"Unhandled pipeline class: {pipeline_name}")

which throws ValueError.

green-anger commented 11 months ago

The workaround for me was renaming the class to DemoFusionStableDiffusionPipeline and adjusting the code accordingly.