Dataloader for evaluation

[MinaGhadimiAtigh]

Hi, great and interesting work. I have a question about the results in the paper. Are the results generated on unlabelled_train_loader or test_loader?

The code only evaluates on the train data loader. As in the function train(model, train_loader, None, test_loader_unlabelled, args) the output is all_acc, old_acc, new_acc = test(student, unlabelled_train_loader, epoch=epoch, save_name='Train ACC Unlabelled', args=args) args.logger.info('Train Accuracies: All {:.4f} | Old {:.4f} | New {:.4f}'.format(all_acc, old_acc, new_acc)) Are the results in the table for the unlabelled_train_loader or test_loader?

There are some lines of code to get the output for the test_loader which are commented.

all_acc_test, old_acc_test, new_acc_test = test(student, test_loader, epoch=epoch, save_name='Test ACC', args=args) args.logger.info('Test Accuracies: All {:.4f} | Old {:.4f} | New {:.4f}'.format(all_acc_test, old_acc_test, new_acc_test))

[RunDevil-Run]

Similar question, is the number in paper using model selection from validation set (labeled class from training set but not overlap to training set) or just simply the last epoch?

[xwen99]

Hi @MinaGhadimiAtigh @RunDevil-Run, my reply to this issue may resolve your questions: https://github.com/sgvaze/generalized-category-discovery/issues/18

The unlabelled_train_loader is where results in the paper are taken from. The test_loader was used for model selection in an earlier version, and now we recommend just using the last epoch (as in the paper).

[MinaGhadimiAtigh]

Thanks a lot for your answer.