search

ruotianluo / self-critical.pytorch

Unofficial pytorch implementation for Self-critical Sequence Training for Image Captioning. and others.
MIT License
996 stars 279 forks source link

Higher Val scores but lower in test #230

Closed kaelsunkiller closed 4 years ago

kaelsunkiller commented 4 years ago

Have you met the similar problem that getting a higher Val score but lower on test split? I trained with the long term new sc configuration two times and got the same results with this problem. Does that mean I have got an over fitting issue? I thought it should be promoted on Val and test split of COCO dataset concurrently from my experience before. This is the first time I met this kind of problem, I wanna ask if it's normal, Thanks!

kaelsunkiller commented 4 years ago

btw, what is the general gap between val and test CIDErs from your experience? I've got model1 with 1.25 on val and 1.30 on test but model2 with 1.27 and 1.29 on val and test respectively. I find in other public models，0.2-0.3 is a normal gap. Is 0.5 normal? Thanks!