luosiallen / latent-consistency-model

Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference
MIT License
4.2k stars 215 forks source link

usage of guidance_scale_embedding as timestamp condtional #58

Closed shuminghu closed 6 months ago

shuminghu commented 7 months ago

Running either train_lcm_distill_sd_wds.py or train_lcm_distill_sdxl_wds.py would running into missing argument error:

Traceback (most recent call last):
  File "/home/smhu/diffusers/examples/consistency_distillation/train_lcm_distill_sd_wds.py", line 1302, in <module>
    main(args)
  File "/home/smhu/diffusers/examples/consistency_distillation/train_lcm_distill_sd_wds.py", line 857, in main
    teacher_unet.config["time_cond_proj_dim"] = args.unet_time_cond_proj_dim
                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Namespace' object has no attribute 'unet_time_cond_proj_dim'

In train_lcm_distill_sd_wds.py, this argument configures guidance_scale_embedding dimension to be used as timestep conditional: https://github.com/luosiallen/latent-consistency-model/blob/main/LCM_Training_Script/consistency_distillation/train_lcm_distill_sd_wds.py#L1143-L1156

, whereas in train_lcm_distill_sdxl_wds.py, guidance_scale_embedding is not used at all. https://github.com/luosiallen/latent-consistency-model/blob/main/LCM_Training_Script/consistency_distillation/train_lcm_distill_sdxl_wds.py#L1248

Is the use of guidance_scale_embedding as timestamp_cond actually helpful? if yes, please fix with a default dim.

Crimson-pro commented 7 months ago
parser.add_argument(
        "--unet_time_cond_proj_dim",
        type=int,
        default=256,
        help=(
            "The dimension of the guidance scale embedding in the U-Net, which will be used if the teacher U-Net"
            " does not have `time_cond_proj_dim` set."
        )

I found this new args in latest diffusers repo, hope could help you