Closed GuohongLi closed 4 years ago
@GuohongLi Thank you very much for asking and sorry for the late reply. As replied in your last issue (https://github.com/zhmiao/OpenLongTailRecognition-OLTR/issues/50#issue-524159914), we finally debugged the published code and current open set performance is:
============ Phase: test
Evaluation_accuracy_micro_top1: 0.361 Averaged F-measure: 0.501 Many_shot_accuracy_top1: 0.442 Median_shot_accuracy_top1: 0.352 Low_shot_accuracy_top1: 0.175
==========
This is higher than we reported in the paper. We updated some of the modules with clone() method, and set use_fc in the first stage to False. These changes will lead us to the proper results. Please have a try. Thank you very much again.
For Places, the current config won't work either. The reason why we could not get the reported results is that we forget that on the first stage, we actually did not freeze the weights. We only freeze the weights on the second stage. We will update the corresponding code as soon as possible.
@GuohongLi Hello, we just updated new configuration files for places, and the newly implemented results are a little better than reported. Please check out the updates. Thanks!
For ImageNet_LT,I just use default config in the code, but cannot reproduce the results in paper Table3(a).
Few/Low shot acc 0.7% is better with 0.4% Plain model in Table3(a) .
[Below is IMPORTANT!!!!!] 2.However for stage2, my result is(some last logs when training complete): Epoch: [60/60] Step: 440 Minibatch_loss_feature: 0.569 Minibatch_loss_performance: 2.938 Minibatch_accuracy_micro: 0.566 Epoch: [60/60] Step: 450 Minibatch_loss_feature: 0.567 Minibatch_loss_performance: 2.845 Minibatch_accuracy_micro: 0.539 Phase: val 100%|██████████| 79/79 [01:34<00:00, 1.02it/s] Phase: val Evaluation_accuracy_micro_top1: 0.340 Averaged F-measure: 0.324 Many_shot_accuracy_top1: 0.401 Median_shot_accuracy_top1: 0.334 Low_shot_accuracy_top1: 0.197 Training Complete. Best validation accuracy is 0.341 at epoch 48
However Many, Median and Few/Low shot acc are 40.1%, 33.4% and 19.7%, which are a little diff with 43.2%, 35.1% and 18.5% in "Ours" model in Table3(a) . And I retrained for several times, the Many-shot acc always some lower than 43.2%.
Are there any tricks not released?