search

ysyscool / SGDNet

ACM MM 2019 SGDNet: An End-to-End Saliency-Guided Deep Neural Network for No-Reference Image Quality Assessment
77 stars 14 forks source link

Try to run the code, but got very low result. #4

Open Zzzbang opened 4 years ago

Zzzbang commented 4 years ago

I try to run your code, but got very low result. Here is my result: ./checkpoint/CSIQ_3/01-0.2828-0.3439-0.2347-0.4749.pkl Testing Results :SROCC: -0.3192 KROCC: -0.2222 PLCC: -0.3327 RMSE: 0.2985 MAE: 0.2347 It stoped at very early epoch. Could you please tell me if i missed someting?

Zzzbang commented 4 years ago

I try to run your code, but got very low result. Here is my result: ./checkpoint/CSIQ_3/01-0.2828-0.3439-0.2347-0.4749.pkl Testing Results :SROCC: -0.3192 KROCC: -0.2222 PLCC: -0.3327 RMSE: 0.2985 MAE: 0.2347 It stoped at very early epoch. Could you please tell me if i missed someting?

I use the params which was detailed in your paper. I used the entire image without sampling. ( In your code, it used sampling with 224 *224) . And other params are unchanged.

Thank you in advance for your assistance.

ysyscool commented 4 years ago

I try to run your code, but got very low result. Here is my result: ./checkpoint/CSIQ_3/01-0.2828-0.3439-0.2347-0.4749.pkl Testing Results :SROCC: -0.3192 KROCC: -0.2222 PLCC: -0.3327 RMSE: 0.2985 MAE: 0.2347 It stoped at very early epoch. Could you please tell me if i missed someting?

I use the params which was detailed in your paper. I used the entire image without sampling. ( In your code, it used sampling with 224 *224) . And other params are unchanged.

Thank you in advance for your assistance.

Can you tell me the config of your training script? I suggest you use my default config to get the similar results in my paper firstly.

Zzzbang commented 4 years ago

I try to run your code, but got very low result. Here is my result: ./checkpoint/CSIQ_3/01-0.2828-0.3439-0.2347-0.4749.pkl Testing Results :SROCC: -0.3192 KROCC: -0.2222 PLCC: -0.3327 RMSE: 0.2985 MAE: 0.2347 It stoped at very early epoch. Could you please tell me if i missed someting?

I use the params which was detailed in your paper. I used the entire image without sampling. ( In your code, it used sampling with 224 *224) . And other params are unchanged. Thank you in advance for your assistance.

Can you tell me the config of your training script? I suggest you use my default config to get the similar results in my paper firstly.

Thank you for your assistance. I re-tried running the code with default config as you suggest. Most of the database could get the similar results in your paper. But TID2013 got relatively low results. Like:

Testing Results : exp: 34 SROCC: 0.6689 KROCC: 0.4977 PLCC: 0.6646 RMSE: 4.2885 MAE: 4.1176

Here is my config of TID2013: TID2013: datainfo: ./tid2013.mat sim_dir: ../data/predictions_DINet_tid2013 im_dir: ../data/tid2013/distorted_images train_ratio: 0.8 test_ratio: 0.2 shape_r: 384 shape_c: 512

ysyscool commented 4 years ago

I try to run your code, but got very low result. Here is my result: ./checkpoint/CSIQ_3/01-0.2828-0.3439-0.2347-0.4749.pkl Testing Results :SROCC: -0.3192 KROCC: -0.2222 PLCC: -0.3327 RMSE: 0.2985 MAE: 0.2347 It stoped at very early epoch. Could you please tell me if i missed someting? I use the params which was detailed in your paper. I used the entire image without sampling. ( In your code, it used sampling with 224 *224) . And other params are unchanged. Thank you in advance for your assistance. Can you tell me the config of your training script? I suggest you use my default config to get the similar results in my paper firstly.

Thank you for your assistance. I re-tried running the code with default config as you suggest. Most of the database could get the similar results in your paper. But TID2013 got relatively low results. Like:

Testing Results : exp: 34 SROCC: 0.6689 KROCC: 0.4977 PLCC: 0.6646 RMSE: 4.2885 MAE: 4.1176

Here is my config of TID2013: TID2013: datainfo: ./tid2013.mat sim_dir: ../data/predictions_DINet_tid2013 im_dir: ../data/tid2013/distorted_images train_ratio: 0.8 test_ratio: 0.2 shape_r: 384 shape_c: 512

In my experience, the intermediate training checkpoint for the TID2013 dataset is better than the last one. But I don't know the reason for this phenomenon.