gwang-kim / DiffusionCLIP

[CVPR 2022] Official PyTorch Implementation for DiffusionCLIP: Text-guided Image Manipulation Using Diffusion Models
Other
785 stars 113 forks source link

tring to reconstrace paper results #21

Open orydatadudes opened 1 year ago

orydatadudes commented 1 year ago

Thank you for your great work i am trying to train ( fine tune ) my own model i am running on CelebA-HQ dataset and i try to train my models for changing a face into Neanderthal/pixar but i get bad results

my configuration for Neanderthal fine tuning: mode = "clip_finetune_eff" #"can't run with clip_finetune - gpu memory error" exp = './runs/finetune_Neanderthal' edit_attr = "neanderthal" n_train_img = 50 n_test_img = 10 n_iter = 50 t_0 = 500 n_inv_step = 40 n_train_step = 6 n_test_step = 40 lr_clip_finetune = 8e-6 id_loss_w = 0 l1_loss_w = 1

the results i get :

after 5 epochs (train):

train_43_2_clip_Neanderthal_5_ngen6

after 8 epochs :

train_43_2_clip_Neanderthal_8_ngen6

after 10 epochs :

train_43_2_clip_Neanderthal_10_ngen6

after 12 epochs : train_37_2_clip_Neanderthal_12_ngen6

same happen for pixar ( change edit_attr = "pixar" ) after 5 epochs:

train_49_2_clip_3D_render_in_the_style_of_Pixar_5_ngen6

after 10 epochs :

train_49_2_clip_3D_render_in_the_style_of_Pixar_10_ngen6

after 15 epochs :

train_7_2_clip_3D_render_in_the_style_of_Pixar_15_ngen6

what i am doing wrong ? thank you

tsa18 commented 1 year ago

Thank you for your great work i am trying to train ( fine tune ) my own model i am running on CelebA-HQ dataset and i try to train my models for changing a face into Neanderthal/pixar but i get bad results

my configuration for Neanderthal fine tuning: mode = "clip_finetune_eff" #"can't run with clip_finetune - gpu memory error" exp = './runs/finetune_Neanderthal' edit_attr = "neanderthal" n_train_img = 50 n_test_img = 10 n_iter = 50 t_0 = 500 n_inv_step = 40 n_train_step = 6 n_test_step = 40 lr_clip_finetune = 8e-6 id_loss_w = 0 l1_loss_w = 1

the results i get :

after 5 epochs (train):

train_43_2_clip_Neanderthal_5_ngen6

after 8 epochs :

train_43_2_clip_Neanderthal_8_ngen6

after 10 epochs :

train_43_2_clip_Neanderthal_10_ngen6

after 12 epochs : train_37_2_clip_Neanderthal_12_ngen6

same happen for pixar ( change edit_attr = "pixar" ) after 5 epochs:

train_49_2_clip_3D_render_in_the_style_of_Pixar_5_ngen6

after 10 epochs :

train_49_2_clip_3D_render_in_the_style_of_Pixar_10_ngen6

after 15 epochs :

train_7_2_clip_3D_render_in_the_style_of_Pixar_15_ngen6

what i am doing wrong ? thank you

Maybe you should set n_iter=1?

wzr1201 commented 6 months ago

have you obtain good results finally, i also get bad results

TerryMelody commented 5 months ago

have you obtain good results finally, i also get bad results

I use "clip_finetune" instead of "clip_finetune_eff", the results are better.