Model 1 results for warping is unexpected

shionhonda / viton-gan

Original implementation of the paper "VITON-GAN: Virtual Try-on Image Generator Trained with Adversarial Loss" by Shion Honda.

https://diglib.eg.org/handle/10.2312/egp20191043

MIT License

101 stars 31 forks source link

Model 1 results for warping is unexpected #4

Open pratiklodha95 opened 5 years ago

pratiklodha95 commented 5 years ago

I implemented the code as mention in this repo and cp-vton repo.

Everything is working fine, but the model results from the part 1 model is not as expected, Would like to if there is a specific formats of input one has to give, I have kept all inputs as per your code and data preparation code from cp-vton repo.

for simple inputs like image

and image

shionhonda commented 5 years ago

Could you give me more information?

How did you obtain warp-person.jpg? Is it a warped cloth layered on the person image?
What did you input? You need cloth mask, parsed person (by LIP) and pose (by OpenPose) in addition to the pair of person and cloth.

Also, I'm not sure about the extrapolation performance of VITON-GAN. The dataset I used only contains female models.

jakubLangr commented 5 years ago

Hi @shionhonda @pratiklodha95 I got the same results. Even on the original test files, using the provided GMM epoch 99 file.

000038_1

Thoughts?

shionhonda commented 5 years ago

Thanks for reporting @jakubLangr .
That is weird because the result in my local repository looks fine. 000038_1

I might have provided different GMM model file. Let me check.

shionhonda commented 5 years ago

I did reproduce the successful result by 1) cloning this repository, 2) downloading the dataset and the trained model gmm_epoch_99.pth, and 3) running python run_gmm.py.
Could you check if the trained model is successfully loaded, please? I'm wondering if TPS transform is really working in your case.

jakubLangr commented 5 years ago

Hi @shionhonda I went ahead and trained your algorithm from scratch. These are the results from my training run over the weekend. The images seem within norms so it seems that the training code for GMM is working on my machine.

However, I am fairly confident that the pretrained epoch_99 checkpoint is used correctly, as I do not get any file not found errors or similar and the path seems to match the requirement.

I do agree with your suggestion that it is likely that TPS is not working, but I am not sure why.

I am now re-running it on the freshly re-trained model.

Out of curiosity, what setup are you using (Pytorch / Cuda versions?) & OS?

jakubLangr commented 5 years ago

On re-run with my re-trained model the results are somewhat better. But I am still confused as to why the neck gets removed?

000010_1

shionhonda commented 5 years ago

@jakubLangr Glad to see that your re-running looks fine. But I'm not sure why as well. I used the setup like:

PyTorch 1.2.0 (it should be fine if > 1.0)
CUDA 9.0
Ubuntu 16.04

Necks are removed in LIP. http://sysu-hcp.net/lip/parsingchallenge.php