search

truskovskiyk / nima.pytorch

NIMA: Neural IMage Assessment
MIT License
330 stars 79 forks source link

why human high score image not got high score in model? #10

Open jt120 opened 6 years ago

jt120 commented 6 years ago

I choose >7 score image, test with model, the scores usually near 5

my test image like these

id score
957587 7.358823529411764
55694 7.393364928909953
552731 7.329787234042553
900122 7.211764705882352
216104 7.5266457680250785
181017 7.225352112676057
254955 7.24776119402985
626541 7.768292682926829
521055 7.232432432432432
60491 7.737804878048781
753064 7.220238095238094
404498 7.770114942528735
911909 7.2706766917293235
571479 7.944954128440367
271660 7.860986547085202
957462 7.275449101796408
128244 7.476340694006308
hcl14 commented 5 years ago

Built histogram of predictions (red, on 0.3 subset of entire set image )

similar to the one in the article: original

It seems this model does not predict mean scores > 7 at all.

hcl14 commented 5 years ago

Btw, how did you get such good variance? I've trained for 50 epochs in tensorflow with the same settings (lr=10-4), imagenet mean and stddev - but it's still a spike:

bs16-low-dropout-ep17