search

zhung2 / uvtranse

10 stars 2 forks source link

Is the model trained using Weighted CE Loss? #2

Closed goelarushi closed 4 years ago

goelarushi commented 4 years ago

Hi @zhung2 ,

Thanks for your repo. When I retrain the model, I am not able to reproduce results in the paper especially for the zero-shot case (Phrase Detection - R@50/R@100 ~ 10.00/13.00). I am using k=1 and cross-entropy loss without weights. Are the results in the paper trained with weighted loss with the 'so_prior.pkl' file? Or any other suggestions for reproducing zero shot results?
Thanks to you in advance.

zhung2 commented 4 years ago

Hello,

We treat k as a hyperparameter for zero-shot case, so it would be better if you try to set k = 5 or k = 10.

Thanks, Zih-Siou

On Mon, May 18, 2020 at 3:20 AM Arushi Goel notifications@github.com wrote:

Hi @zhung2 https://github.com/zhung2 ,

Thanks for your repo. When I retrain the model, I am not able to reproduce results in the paper especially for the zero-shot case (Phrase Detection - R@50/R@100 ~ 10.00/13.00). I am using k=1 and cross-entropy loss without weights. Are the results in the paper trained with weighted loss with the 'so_prior.pkl' file? Or any other suggestions for reproducing zero shot results? Thanks to you in advance.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/zhung2/uvtranse/issues/2, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKU56TNTGBGTEDOQTGZITPDRSDVVJANCNFSM4ND3JDJQ .

goelarushi commented 4 years ago

That might solve my problem! Thanks a lot for your response @zhung2 .

alice-cool commented 3 years ago

Hi @zhung2 ,

Thanks for your repo. When I retrain the model, I am not able to reproduce results in the paper especially for the zero-shot case (Phrase Detection - R@50/R@100 ~ 10.00/13.00). I am using k=1 and cross-entropy loss without weights. Are the results in the paper trained with weighted loss with the 'so_prior.pkl' file? Or any other suggestions for reproducing zero shot results? Thanks to you in advance.

Dear scholar， I run train.py but it throws errors. Some files the website didn't provide. Could you share some files? Or provide download link? Thank you The two model parameter image The file image The file image The file image

kent252 commented 1 year ago

That might solve my problem! Thanks a lot for your response @zhung2 .

Hello @goelarushi I got the same problem as you, could you show me your config or add any file to get the paper result. I appreciate your help. Thank you.