search

adobe-research / custom-diffusion

Custom Diffusion: Multi-Concept Customization of Text-to-Image Diffusion (CVPR 2023)
https://www.cs.cmu.edu/~custom-diffusion
Other
1.84k stars 137 forks source link

How to train the model withot using the regularized dataset #69

Open Aditi0912dec opened 1 year ago

Aditi0912dec commented 1 year ago

Can you please let me know how to train the model without using the regularized dataset. In train.py there is an argument --repeat, will only enabling that work. Thank you.

nupurkmr9 commented 1 year ago

Hi, You would also need to remove the reg_datapath and reg_caption related arguments and adding repeat:100 to the data.params.train.params in the config file. Thus the training command for a single concept would look like this along with the changes in the finetune_addtoken.yaml config file:

python -u train.py \ --base configs/custom-diffusion/finetune_addtoken.yaml \ -t --gpus 0,1 \ --resume-from-checkpoint-custom \ --caption " cat" \ --datapath \ --modifier_token "" \ --name "cat-sdv4"

There would be similar changes for the multi-concept case as well. Let me know if this resolves your issue. Thanks.

Aditi0912dec commented 1 year ago

Thank you for the help. It worked.

I have just one more doubt when I am trying to convert the ckpt model file into delta bin it is giving an error stating ImportError: cannot import name 'CLIPTextModelWithProjection' from 'transformers'. The version of the transformer is 4.19.2.

Sorry to bother you, but can you let me know what can be the solution? Thank you

Regards Aditi Palit Ph.D. scholar Computer Science and Engineering Department IIT Tirupati

On Fri, Jul 21, 2023 at 9:23 PM nupurkmr9 @.***> wrote:

Hi, You would also need to remove the reg_datapath and reg_caption related arguments and adding repeat:100 to the data.params.train.params in the config file. Thus the training command for a single concept would look like this along with the changes in the finetune_addtoken.yaml config file:

python -u train.py --base configs/custom-diffusion/finetune_addtoken.yaml -t --gpus 0,1 --resume-from-checkpoint-custom --caption " cat" --datapath --modifier_token "" --name "cat-sdv4"

There would be similar changes for the multi-concept case as well. Let me know if this removes your issue. Thanks.

— Reply to this email directly, view it on GitHub https://github.com/adobe-research/custom-diffusion/issues/69#issuecomment-1645794826, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASR5NWBJY373NL5IMJRLC23XRKQXJANCNFSM6AAAAAA2RIWK7A . You are receiving this because you authored the thread.Message ID: @.***>