large feature map parameter settings and CUB MaxboxaccV2 score matters

[dongjunhwang]

great work. thank you for sharing this source. But i have a little question about the parameter of large feature map and boxaccv2 score in CUB dataset.

1. I found that large_feature_map parameter is False only the resnet50 architecture. Is that right settings?

2. In CUB dataset, i evaluated to use the checkpoint that you opened in google drive. But the score is different to compare the paper.

The paper show the boxaccv2 score for CUB dataset to 71.35 in resnet backbone, but evaluation result is represented like this.

Evaluation on test set ...
Evaluate epoch 0, split test
Computing and evaluating cams.
Split val, metric Top-1 Cls, current value: 74.0
Split val, metric Top-1 Cls, best value: 74.0
Split val, metric Top-1 Cls, best epoch: 0
Split val, metric GT-Known Loc, current value: 76.8
Split val, metric GT-Known Loc, best value: 76.8
Split val, metric GT-Known Loc, best epoch: 0
Split val, metric Top-1 Loc, current value: 60.0
Split val, metric Top-1 Loc, best value: 60.0
Split val, metric Top-1 Loc, best epoch: 0
Split val, metric MaxBoxAccV2, current value: 62.93333333333333
Split val, metric MaxBoxAccV2, best value: 62.93333333333333
Split val, metric MaxBoxAccV2, best epoch: 0
Split test, metric Top-1 Cls, current value: 79.04729030031066
Split test, metric GT-Known Loc, current value: 81.24137931034483
Split test, metric Top-1 Loc, current value: 69.51724137931035

# Wrong Part
Split test, metric MaxBoxAccV2, current value: 68.64367816091954

The other dataset have the right score, but only CUB datasets are wrong.

Shell setting is like below.

CUDA_VISIBLE_DEVICES=1 python main.py --data_root $data_root \
                --experiment_name CUB_MMD_CAM_RES \
                --pretrained TRUE \
                --num_val_sample_per_class 0 \
                --large_feature_map True \
                --batch_size 32 \
                --epochs 10 \
                --lr 1E-3 \
                --lr_decay_frequency 3 \
                --weight_decay 1.00E-04 \
                --override_cache FALSE \
                --workers 16 \
                --box_v2_metric True \
                --iou_threshold_list 30 50 70 \
                --eval_checkpoint_type last \
                --has_grid_size 111 \
                --has_drop_rate 0.19 \
                --cutmix_beta 1.35 \
                --cutmix_prob 0.34 \
                --adl_threshold 0.99 \
                --adl_drop_rate 0.59 \
                --save_dir 'test_logs' \
                --seed 4 \
                --dataset_name CUB \
                --architecture resnet50 \
                --wsol_method cam \
                --uda_method mmd \
                --beta 0.3 \
                --univer 2 \
                --start_epoch 0 \
                --check_path "dawsol_cub_res.pth.tar" \
                --mode "test"

I want to reproduce this code only for evaluation, so it seems little important to me. Is that checkpoint have the right value?

Thanks.

[zh460045050]

Thank you for your attention on our work.

For your questions:

1. We set large_feature_map is False for only for the resnet50 architecture on OpenImages dataset. For other cases, we set large_feature_map=True.

2. Thank you for your question. We have checked it and found that we uploaded a wrong checkpoint for CUB-200 dataset on our google drive. Now, the right checkpoint has been already uploaded in our google drive ("dawsol_cub_res_update.tar"). You can re-download it and run with your shell setting (this setting is right).