Open mlkk518 opened 4 years ago
HuiZeng
commented
4 years ago Hi, I has been several years that I did not work on the IQA topic. If I remember correctly, the variance is very large when using different random splits. That may be the main reason. You can use more repetitions to check it.
mlkk518
commented
4 years ago Thanks a lot for your reply quickly , I would try the next.
------------------ 原始邮件 ------------------ 发件人: "Hui Zeng"<notifications@github.com>; 发送时间: 2020年6月4日(星期四) 下午3:45 收件人: "HuiZeng/BIQA_Toolbox"<BIQA_Toolbox@noreply.github.com>; 抄送: "1198484350"<mlkkljh@qq.com>;"Author"<author@noreply.github.com>; 主题: Re: [HuiZeng/BIQA_Toolbox] LIVE dataset test result (#6)
Hi, I has been several years that I did not work on the IQA topic. If I remember correctly, the variance is very large when using different random splits. That may be the main reason. You can use more repetitions to check it.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.
Hello! Sorry to bother you! I run your code without any changes except for increasing the epoch from 10 to 20 and reducing the number of repetition from 10 to 2. However, what makes me upset is that I could hardly get the same metric values which just near yours ,such as PLCC and SROCC. Both LIVE dataset and ResNet network are choosed for the experienment. Here is the results: `( Parameters: LIVE; ResNet; bins = 5; patches = 50; SRCC: 0.9478; 0.9494; 0.9514; 0.9522; 0.9527; 0.9509; 0.9510; 0.9509; 0.9513; 0.9521; std_SRCC: 0.0031; 0.0005; 0.0006; 0.0030; 0.0012; 0.0012; 0.0003; 0.0005; 0.0015; 0.0007; PLCC: 0.9556; 0.9570; 0.9587; 0.9589; 0.9602; 0.9582; 0.9589; 0.9581; 0.9589; 0.9593; std_PLCC: 0.0057; 0.0032; 0.0051; 0.0073; 0.0047; 0.0054; 0.0046; 0.0047; 0.0058; 0.0051; best_SRCC: 0.9527; best_PLCC: 0.9602
Parameters: LIVE; ResNet; bins = 5; patches = 50; SRCC: 0.9457; 0.9457; 0.9503; 0.9504; 0.9522; 0.9514; 0.9510; 0.9510; 0.9517; 0.9527; 0.9523; 0.9530; 0.9526; 0.9518; 0.9512; 0.9525; 0.9515; 0.9518; 0.9520; 0.9524; std_SRCC: 0.0004; 0.0045; 0.0004; 0.0004; 0.0007; 0.0022; 0.0013; 0.0006; 0.0008; 0.0009; 0.0000; 0.0011; 0.0002; 0.0008; 0.0010; 0.0011; 0.0000; 0.0003; 0.0005; 0.0022; PLCC: 0.9547; 0.9551; 0.9572; 0.9575; 0.9593; 0.9579; 0.9581; 0.9577; 0.9579; 0.9587; 0.9589; 0.9590; 0.9594; 0.9584; 0.9586; 0.9590; 0.9580; 0.9584; 0.9583; 0.9587; std_PLCC: 0.0032; 0.0001; 0.0036; 0.0042; 0.0025; 0.0050; 0.0029; 0.0031; 0.0045; 0.0032; 0.0039; 0.0029; 0.0028; 0.0033; 0.0020; 0.0023; 0.0035; 0.0031; 0.0033; 0.0050; best_SRCC: 0.9530; best_PLCC: 0.9590 `)