search

zhuhancheng / MetaIQA

126 stars 27 forks source link

low srocc when training quality prior model #6

Open HC-2016 opened 4 years ago

HC-2016 commented 4 years ago

Thanks for your great work! I get low srocc on test set ChallengeDB(LIVE_WILD) after training on tid2013 and kadid10k in the first step to train quality prior model.

############# TID 2013 train phase epoch 49 ############### current loss = 0.006955859042742911 ############# Kadid train phase epoch 49 ############### current loss = 0.012211388528347016 ############# test phase epoch 49 ############### ----worker number: 0---- 1 new srocc 0.346739, best srocc 0.403438

I didn't make any change and can you give any suggestions?

zhuhancheng commented 4 years ago

The trained quality prior model should be fine-tuned to test on the LIVE Challenge database.

HC-2016 commented 4 years ago

Get. Thank you for your reply. However SROCC is only related to the rank. Does this mean the model generalizes poorly cross different dataset?

zhuhancheng commented 4 years ago

Our method requires fine-tuning, and it is indeed hard to test the authentically distorted image directly using the prior model trained by synthetic distortions.