Low results on ITR and TIR

[GaryHuangJL]

❓ Questions and Help

I have downloaded pretrained models VQVAE Your pretrained checkpoint

Run this command

python mmf_cli/run.py \ config=projects/fashionvil/configs/e2e_contrastive.yaml \ model=fashionvil \ dataset=fashiongen \ run_type=train_val \

After fine-tune,I ran the test program python mmf_cli/run.py \ config=projects/fashionvil/configs/e2e_contrastive.yaml \ model=fashionvil \ dataset=fashiongen \ run_type=test \ checkpoint.resume_file=save/fashionvil_contrastive_fashiongen_e2e_pretrain_final/best.ckpt

Results test/fashiongen/contrastive_loss: 0.0000, test/total_loss: 0.0000, test/fashiongen/r@k_general/i2t_r@1: 0.3972, test/fashiongen/r@k_general/i2t_r@5: 0.6867, test/fashiongen/r@k_general/i2t_r@10: 0.7836, test/fashiongen/r@k_general/t2i_r@1: 0.4759, test/fashiongen/r@k_general/t2i_r@5: 0.7278, test/fashiongen/r@k_general/t2i_r@10: 0.8226, test/fashiongen/r@k_general/avg: 0.6489

I am wondering What went wrong Thanks you very much

[BrandonHanx]

How you did fine-tune?

[GaryHuangJL]

As the procedure you gave

[BrandonHanx]

More info is needed. Do you have log file? I cannot help you with such limited information.

[GaryHuangJL]

https://drive.google.com/file/d/1cgDL7jWVtr-HGMveZmuUtPfagzd4r7rc/view?usp=sharing This is my training log

[BrandonHanx]

Didn't find errors in your log.

But I re-tested my model, I got the following results:

Here is my model. Can you try it and see whether we have the same result?

[GaryHuangJL]

Thank you for your answering！I will try it later

[GaryHuangJL]

Hi！I use the pretrained weight you gave and run the command as follow: CUDA_VISIBLE_DEVICES=0 python mmf_cli/run.py config=projects/fashionvil/configs/e2e_contrastive.yaml model=fashionvil dataset=fashiongen run_type=test checkpoint.resume_file=save/fashionvil_contrastive_fashiongen_e2e_pretrain_final/fashionvil_contrastive_final.pth But only get such result What went wrong

[GaryHuangJL]

And this is my test log https://drive.google.com/file/d/1oDvKgfJCtSqq1rz5kiG-qCXRPDGSbSWe/view?usp=sharing

[BrandonHanx]

I didn't find errors in your logs at this moment.

Can you show me your FashionGen dataset structure? And also how you download and preprocess FashionGen.

[GaryHuangJL]

This is my FashionGen dataset structure and i download and extract the FashionGen dataset from these two links below https://drive.google.com/file/d/1yR-_NJ6CxCYB2i8UECEB2doNssj5J8Li/view https://drive.google.com/file/d/1R4JtSAtbbOi9mByBj-KpchNf9nDn6BO1/view which from the project: https://github.com/menardai/FashionGenAttnGAN

[BrandonHanx]

Can you randomly choose some images from your train and val folders?

I want to check if the pic names are aligned between us.

[GaryHuangJL]

in train folder： picture name：115406.jpg picture name: 163638.jpg picture name:253890.jpg in val folder: picture name:12749.jpg picture name :20477.jpg picture name:28723.jpg Please help me check if there are any mistakes in my dataset Thank you!

[GaryHuangJL]

Sorry, i find that all my pictures are in jpg format instead of png format.Is that cause the low result? But i still want you to help me to find beside the format,if the picture names are aligned between us

[GaryHuangJL]

I re-extracted the image to png format and tested it.I got the result as the above.The same as yours. Apologize for my carelessness!