Open hcl14 opened 5 years ago
I don't have the resources to train these models so I won't be able to improve them.
I wonder whether the difference lies in the loss function or the amount of training.
I did not succeed in training with SGD with lr=10-7 for base net and 10-6 for last layer, as authors. It just does not converge. I try to use Adam with oversampling of underrepresented images with mean <4 and >7, but no success, I just get thin shifted spike for variance and something like your picture for mean.
I kind of guessed that the provided learning rates in the paper were too low to be of any use, which is why I switch to Adam with higher learning rates.
I have to analyse the repository you posted, to see what the difference is between my implementation and theirs.
They seem to use Adam with following parameters:
"batch_size": 96,
"epochs_train_dense": 5,
"learning_rate_dense": 0.001,
"decay_dense": 0,
"epochs_train_all": 9,
"learning_rate_all": 0.00003,
"decay_all": 0.000023,
Did not study closely though. Will try to replicate this for my MobileNetV2
This keras implementation: https://github.com/truskovskiyk/nima.pytorch They do adjusting images with ImageNet mean and variance, and use Adam with lr=1e-4.
Histogram is built on 0.3 subset of entire set.
Histograms of the ground truth and predicted scores from the article, p.7
And did the same for two models here:
It shows that MobileNet will show correct scores for very few ground truth images with score <4 and > 7.
Better one is this implementation (mobilenet): https://github.com/idealo/image-quality-assessment
Actually, I'm having troubles myself trying to fit MobileNet2, I'm getting something similar to your mobilenet image.
My histograms are bult on 0.1 subset of entire set.