Open Marshall-yao opened 5 years ago
The evaluation for perceptual SR methods is still difficult. There are several ways:
So, there is still no one faithful metric for perceptual SR. Usually, we examine all of these metrics and all these metrics give a side view of the algorithm performance. In practice during training, I usually visualize some 'typical' regions of selected images. Though bias will be introduced, it somehow gives a direct evaluation of whether the model is good or not.
thanks so much for your patient reply.
I have heard of a classification of results to evaluate the performance of the GAN method. What do you think about this method?
By saying ' visualize some 'typical' regions of selected images', which images do you usually choose and which regions of the image are selected? For example, the selected regions are hair, grass, beard and other areas.
There is a disadvantage to the method of judging the performance of a model by visualizing the results of the model. If the difference between the visual effect of the improved method and the original method is not obvious, it is difficult to distinguish the quality of the improved model.
I think , in this case, it should judge by means of PSNR or perceptual index. What is your opinion of this problem?
Thanks so much.
Hi,Xintao. Excuse me,After reading SFTGAN, ESRGAN, and RankGAN papers, i would like to discuss with you about how to evaluate the reconstruction effect of the GAN method.
1)SFTGAN uses the method of user evaluation toevaluate reconstruction effect. This is not as convincing as the objective evaluation criteria, and may be rejected by the reviewers.
2)ESRGAN uses standard test sets to test PSNR and SSIM . And the test results are very high and refreshed. This clearly illustrates the effectiveness of the method used and is more convincing to the reviewer.
3)RankSRGAN uses NIQE and other evaluation metrics that are more suitable for the GAN method.
If I want to use SFTGAN as the baseline (running time considerations), based on the above considerations, should I use the NIQE evaluation method ? Is subjective evaluation necessary? Are there other evaluation methods?
Best regards.