Learning Rate scheduler

[vinm007]

Hi, Thanks for sharing this amazing work. I have been using this hugging face-based repo for fine-tuning - https://github.com/huggingface/instruction-tuned-sd but I have some queries regarding the learning rate scheduler. In the paper, it was mentioned - We use a learning rate of 10−4 (without any learning rate warm up). Is it constant or some scheduler is used, I do see some config in train.yaml but can't understand

    scheduler_config: # 10000 warmup steps
      target: ldm.lr_scheduler.LambdaLinearScheduler
      params:
        warm_up_steps: [ 0 ]
        cycle_lengths: [ 10000000000000 ] # incredibly large number to prevent corner cases
        f_start: [ 1.e-6 ]
        f_max: [ 1. ]
        f_min: [ 1. ]

If it's being used, what would be the equivalent of this scheduler on diffusers.