tumurzakov / AnimateDiff

AnimationDiff with train
Apache License 2.0
111 stars 28 forks source link

Multiple changes #10

Closed tumurzakov closed 1 year ago

tumurzakov commented 1 year ago
  1. Train AnimateDiff (24+ frames by multiplying existing module by scale factor and finetune)

    # Multiply pe weights by multiplier for training more than 24 frames
    if motion_module_pe_multiplier > 1:
        for key in motion_module_state_dict:
          if 'pe' in key:
            t = motion_module_state_dict[key]
            t = repeat(t, "b f d -> b (f m) d", m=motion_module_pe_multiplier)
            motion_module_state_dict[key] = t 

    I trained till 264 frames on A100

  2. Train AnimateDiff + LoRA

  3. Infinite infer (credits to dajes) (temporal_context and video_length params).

  4. ControlNet (works with Infinite infer). VRAM consumming. Can only infer 120 frames on single controlnet module on A100

  5. Prompt Walking. Start from Egg and finish with Duck

    {
     0: "Egg",
     10: "Duck",
    }
  6. Updated to last diffusers version