search

zwx8981 / UNIQUE

The repository for 'Uncertainty-aware blind image quality assessment in the laboratory and wild' and 'Learning to blindly assess image quality in the laboratory and wild'
Apache License 2.0
127 stars 10 forks source link

Training UNIQUE on single dataset #3

Closed pencilzhang closed 3 years ago

pencilzhang commented 3 years ago

Hi, I have a question about Table VII in your recent arxiv paper. I noticed that you did single-dataset training on baseline model (regression) to compare with UNIQIE model trained on multiple datasets, as shown in Table VII. Why did not train UNIQUE model on one single dataset? Thanks in advance!

zwx8981 commented 3 years ago

@pencilzhang The main advantage of UNIQUE over traditional training strategies is it enables a BIQA model to be trained on multiple databases under a learning-to-rank framework. In our preliminary experiment, we tried training UNIQUE on a single dataset, yet we find that the proposed method brings no benefit to single-dataset training. It is a reasonable result since training by ranking alone does not introduce any extra information compared with training with regression. In our ablation study, we use regression to obtain the results because the training time is significantly lower than the learning-to-rank method. Note that we sample a large number of image pairs from a dataset.

pencilzhang commented 3 years ago

@zwx8981 Thanks for your quick reply! 1) When you say "we tried training UNIQUE on a single dataset, yet we find that the proposed method brings no benefit to single-dataset training.", is learning-by-regression compared with the proposed method in this case? 2) In your work, image pairs are sampled from individual dataset (intra-dataset setting). I think the main contriburion is from more data. Have you ever tried to train learning-by-regression method (baseline) using all databases? 3) I think if you could release pre-trained model that would be very helpful to the community:)

zwx8981 commented 3 years ago

@pencilzhang 1，Yes, we compare it with learning-by-regression. 2，Yes, see the ablation study. The linear re-scaling means we linearly re-scale the MOS of all databases into a unified scale (0-100)，then we combine the images from all databases and train the model using the learning-by-regression baseline. 3, Thank you for the suggestion. We will release the pre-trained model once the paper is accepted.