johannakarras / DreamPose

Official implementation of "DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion"
MIT License
962 stars 73 forks source link

Wrong demo output #38

Open laiyingxin2 opened 1 year ago

laiyingxin2 commented 1 year ago

I used an image from your tutorial finetune and got something like this. I downloaded the Demo and put the weights in a fixed location,but got this ,the face is very blur

1.Finetune the UNet

accelerate launch finetune-unet.py --pretrained_model_name_or_path="CompVis/stable-diffusion-v1-4" --instance_data_dir=demo/sample/train --output_dir=demo/custom-chkpts --resolution=512 --train_batch_size=1 --gradient_accumulation_steps=1 --learning_rate=1e-5 --num_train_epochs=500 --dropout_rate=0.0 --custom_chkpt=checkpoints/unet_epoch_20.pth --revision "ebb811dd71cdc38a204ecbdd6ac5d580f529fd8c"

2.Finetune the VAE decoder accelerate launch --num_processes=1 finetune-vae.py --pretrained_model_name_or_path="CompVis/stable-diffusion-v1-4" --instance_data_dir=demo/sample/train --output_dir=demo/custom-chkpts --resolution=512 --train_batch_size=4 --gradient_accumulation_steps=4 --learning_rate=5e-5 --num_train_epochs=1500 --run_name finetuning/ubc-vae --revision "ebb811dd71cdc38a204ecbdd6ac5d580f529fd8c"

3.Testing python test.py --epoch 499 --folder demo/custom-chkpts --pose_folder demo/sample/poses --key_frame_path demo/sample/key_frame.png --s1 8 --s2 3 --n_steps 100 --output_dir results --custom_vae demo/custom-chkpts/vae_1499.pth

) e9c0a80c-8dea-4a1a-beb5-7a69ada9d7c7

5dc61513-3d09-4eee-8e8b-e0052ee2cdd3 4ca745f6-07d4-4ace-92a1-d53f9eb35bd3

Fanghaipeng commented 1 year ago

i also have the same problem