result recurrence problem

[aishangcengloua]

When I reproduced your experimental results, I found that the performance of the models was quite different from yours. When I experimented on the Kather dataset, only using 1% training samples to train Baseline+FC * and test on the three models. The accuracy in the test set reached 0.51, 0.86 and 0.88, much higher than that in the paper, but these results were very close to that in the LC25000 dataset. I don't know whether the results of the two datasets were written backwards in the paper

[aishangcengloua]

I strictly follow your steps and settings, and guarantee that I have not added any tricks

[linjiatai]

@aishangcengloua

Thank you for your interest in our work. My guess is that a difference in model state setting cause this problem.

It is mentioned in section V.A of our paper that “(1) Froze the baseline models pre-trained on ImageNet and only update the fully connected layers (FC), denoted as Baseline + FC*”.

In order to compare the performances of plugging our PDBL and only updating FC layer under an exact same feature extractor/CNN backbone, we should freeze the CNN backbone completely to be fair.

However, “model.train()” state can not freeze the CNN backbone completely due to the affect of BN layers and Dropout layers. Hence, we set the model into “model.eval()” state in the Baseline+FC* experiments and specify that the optimizer only updates parameters of the FC layer:

If args.model_name == ‘eff’:
        Optimizer = SGD(model._fc.parameters(), lr=0.001, momentum=0.9, dampening=0, weight_decay=0.0001)
else:
        Optimizer = SGD(model.fc.parameters(), lr=0.001, momentum=0.9, dampening=0, weight_decay=0.0001)

In a word, we set the model into the “model.train ()” state for training the whole model, but we set it into the “model.eval ()” state for updating the Baseline+FC* model in our experiments.

Note that “model.eval()” state dose not affect the “_requiresgrad” state and gradient descent.

[aishangcengloua]

Thank you very much for your patience. :smile:

I really set the model into "model.train()" state when I update the Baseline+FC* model. It's probably why I get such a high accuracy. I'll retrain later.

Another problem is that how many epochs I should set up when I update Baseline+FC* model. I train 50 epochs in the above results.