search

adobe-research / custom-diffusion

Custom Diffusion: Multi-Concept Customization of Text-to-Image Diffusion (CVPR 2023)
https://www.cs.cmu.edu/~custom-diffusion
Other
1.87k stars 139 forks source link

Multiple ckpt after training. Which one should I use for sampling? #55

Closed kasim0226 closed 1 year ago

kasim0226 commented 1 year ago

Thank you for your great work.

After I exucted finetune_real.sh and get_deltas.py, I got 4 different ckpt.

  1. Which ckpt should I use for sampling?
  2. Could you teach me why you need to updated model weights with get_deltas.py? Because I thought model weights are trained well with eq. 4 in your paper.
nupurkmr9 commented 1 year ago

Hi, if you followed our readme instructions, you can sample using delta_epoch=000004.ckpt according to the below command: 

python sample.py --prompt "<new1> cat playing with a ball" --delta_ckpt logs/<folder-name>/checkpoints/delta_epoch\=000004.ckpt --ckpt <pretrained-model-path>

We save checkpoints every epoch of training. To answer your second question, get_deltas.py extracts and saves the subset of parameters that were fine-tuned instead of full model weights, thus reducing storage requirements to 75MB instead of the 4 GB original model.

I hope this clarifies your doubts. Thanks.

kasim0226 commented 1 year ago

Thank you for the reply.