chcorbi
commented
2 years ago Hi @dubowsky ,
Thank you for your interest in our work !
-
I check on my side and also obtain 99,88% train accuracy for this run. The reported results in Table 3 was an average accuracy for 5 runs, which might explain the difference. Indeed, the number of errors is important to learn ConfidNet, you should try as much as possible to overfit, adding some regularization might help (see also answer to 2.)
-
At the time of the paper, we used an Adam optimizer with LR 1e-3 but you can use any other parameter setting that converge as well, such as SGD with momentum and multi-step decay scheduler. In the end, what will be important is to avoid too much to overfit, in order to have enough errors in training to learn ConfidNet.
Hope this will help for your project :)
Hope
dubowsky
Hi, thanks for sharing the really nice work and codes!
When running the code in cifar10 with the given pre-trained VGG model, I have come across two problems:
I can get the reported result with the pre-trained original classification model(92.20%). But I found the training accuracy for the pre-trained original classification model is 99.9% which is a little bit different from Table 3(98.69%) in the supplementary. As the error number in the training set seems important for the latter confidence training according to Table 3 in the paper, so I wonder if there is any problem with my computed training accuracy or my understanding is incorrect.
I found the parameters config in the given pre-trained classification model setting is slightly different from the paper description, e.g., Adam vs SGD. Is there any recommended parameter setting to get a similar baseline classifier for cifar10 in the paper?