CAM performance in table 4

[Zhengyang1995]

Hey, thank you for your work. I just have a question for CAM performance in table 4, the miou value is 50.1%. Since recent methods based on Resnet50 and vgg16 only reach their CAM among 48%, if you add any nouveau/extra process to reach that performance?

By the way, maybe there is a miss in 4.3, compared with original one, IPE improve the CAM by 3.1%(which is 2.1% in paper)

[chenqi1126]

We use the official pre-trained model provided by Pytorch and fixed the batchnorm layer following the common practice. You can find here.

[Zhengyang1995]

Sorry for bothering, however, followed by your link, it is code from irn. I could only get 48.6 for CAM performance, which is close to irn paper(48.3 for CAM). Would you please rethink if there is any other process for CAM, since it is so hard to improve to 50.1% for only CAM.

[chenqi1126]

@Zhengyang1995,

I have run the original scripts from irn just now, and this is my training logs:

{'num_workers': 56, 'voc12_root': '../PascalVOC2012/VOCdevkit/VOC2012', 'train_list': 'voc12/train_aug.txt', 'val_list': 'voc12/val.txt', 'infer_list': 'voc12/train_aug.txt', 'chainer_eval_set': 'train', 'cam_network': 'net.resnet50_cam', 'cam_crop_size': 512, 'cam_batch_size': 16, 'cam_num_epoches': 5, 'cam_learning_rate': 0.1, 'cam_weight_decay': 0.0001, 'cam_eval_thres': 0.11, 'cam_scales': (1.0, 0.5, 1.5, 2.0), 'conf_fg_thres': 0.3, 'conf_bg_thres': 0.05, 'irn_network': 'net.resnet50_irn', 'irn_crop_size': 512, 'irn_batch_size': 32, 'irn_num_epoches': 3, 'irn_learning_rate': 0.1, 'irn_weight_decay': 0.0001, 'beta': 10, 'exp_times': 8, 'ins_seg_bg_thres': 0.25, 'sem_seg_bg_thres': 0.25, 'log_name': 'sample_train_eval', 'cam_weights_name': 'sess/res50_cam.pth', 'irn_weights_name': 'sess/res50_irn.pth', 'cam_out_dir': 'result/cam', 'ir_label_out_dir': 'result/ir_label', 'sem_seg_out_dir': 'result/sem_seg', 'ins_seg_out_dir': 'result/ins_seg', 'train_cam_pass': True, 'make_cam_pass': True, 'eval_cam_pass': True, 'cam_to_ir_label_pass': False, 'train_irn_pass': False, 'make_ins_seg_pass': False, 'eval_ins_seg_pass': False, 'make_sem_seg_pass': False, 'eval_sem_seg_pass': False}

step.train_cam: Mon Oct 17 15:38:54 2022
Epoch 1/5
step:    0/ 3305 loss:0.6748 imps:1.5 lr: 0.1000 etc:Tue Oct 18 01:38:48 2022
step:  100/ 3305 loss:0.1818 imps:90.1 lr: 0.0973 etc:Mon Oct 17 15:48:45 2022
step:  200/ 3305 loss:0.1114 imps:128.9 lr: 0.0945 etc:Mon Oct 17 15:45:48 2022
step:  300/ 3305 loss:0.0997 imps:150.9 lr: 0.0918 etc:Mon Oct 17 15:44:48 2022
step:  400/ 3305 loss:0.0912 imps:164.7 lr: 0.0890 etc:Mon Oct 17 15:44:19 2022
step:  500/ 3305 loss:0.0848 imps:174.4 lr: 0.0863 etc:Mon Oct 17 15:44:01 2022
step:  600/ 3305 loss:0.0848 imps:181.6 lr: 0.0835 etc:Mon Oct 17 15:43:49 2022
validating ... loss: 0.0708
Epoch 2/5
step:  700/ 3305 loss:0.0780 imps:75.5 lr: 0.0807 etc:Mon Oct 17 15:45:25 2022
step:  800/ 3305 loss:0.0700 imps:144.2 lr: 0.0779 etc:Mon Oct 17 15:45:05 2022
step:  900/ 3305 loss:0.0746 imps:169.8 lr: 0.0751 etc:Mon Oct 17 15:44:51 2022
step: 1000/ 3305 loss:0.0698 imps:182.8 lr: 0.0723 etc:Mon Oct 17 15:44:39 2022
step: 1100/ 3305 loss:0.0666 imps:190.8 lr: 0.0695 etc:Mon Oct 17 15:44:29 2022
step: 1200/ 3305 loss:0.0679 imps:196.4 lr: 0.0666 etc:Mon Oct 17 15:44:21 2022
step: 1300/ 3305 loss:0.0658 imps:200.9 lr: 0.0638 etc:Mon Oct 17 15:44:14 2022
validating ... loss: 0.0662
Epoch 3/5
step: 1400/ 3305 loss:0.0551 imps:113.8 lr: 0.0609 etc:Mon Oct 17 15:44:43 2022
step: 1500/ 3305 loss:0.0565 imps:157.6 lr: 0.0580 etc:Mon Oct 17 15:44:36 2022
step: 1600/ 3305 loss:0.0550 imps:176.6 lr: 0.0551 etc:Mon Oct 17 15:44:29 2022
step: 1700/ 3305 loss:0.0566 imps:187.0 lr: 0.0522 etc:Mon Oct 17 15:44:24 2022
step: 1800/ 3305 loss:0.0574 imps:193.8 lr: 0.0493 etc:Mon Oct 17 15:44:19 2022
step: 1900/ 3305 loss:0.0569 imps:198.8 lr: 0.0463 etc:Mon Oct 17 15:44:14 2022
validating ... loss: 0.0542
Epoch 4/5
step: 2000/ 3305 loss:0.0549 imps:40.2 lr: 0.0433 etc:Mon Oct 17 15:44:35 2022
step: 2100/ 3305 loss:0.0482 imps:132.3 lr: 0.0403 etc:Mon Oct 17 15:44:30 2022
step: 2200/ 3305 loss:0.0488 imps:163.3 lr: 0.0373 etc:Mon Oct 17 15:44:26 2022
step: 2300/ 3305 loss:0.0472 imps:178.5 lr: 0.0343 etc:Mon Oct 17 15:44:22 2022
step: 2400/ 3305 loss:0.0437 imps:187.8 lr: 0.0312 etc:Mon Oct 17 15:44:18 2022
step: 2500/ 3305 loss:0.0482 imps:194.0 lr: 0.0281 etc:Mon Oct 17 15:44:15 2022
step: 2600/ 3305 loss:0.0466 imps:198.7 lr: 0.0249 etc:Mon Oct 17 15:44:12 2022
validating ... loss: 0.0519
Epoch 5/5
step: 2700/ 3305 loss:0.0439 imps:93.8 lr: 0.0217 etc:Mon Oct 17 15:44:27 2022
step: 2800/ 3305 loss:0.0416 imps:148.9 lr: 0.0184 etc:Mon Oct 17 15:44:24 2022
step: 2900/ 3305 loss:0.0385 imps:171.2 lr: 0.0151 etc:Mon Oct 17 15:44:21 2022
step: 3000/ 3305 loss:0.0408 imps:183.4 lr: 0.0117 etc:Mon Oct 17 15:44:18 2022
step: 3100/ 3305 loss:0.0386 imps:191.0 lr: 0.0082 etc:Mon Oct 17 15:44:15 2022
step: 3200/ 3305 loss:0.0358 imps:196.2 lr: 0.0045 etc:Mon Oct 17 15:44:13 2022
step: 3300/ 3305 loss:0.0383 imps:200.5 lr: 0.0003 etc:Mon Oct 17 15:44:10 2022
validating ... loss: 0.0490
step.make_cam: Mon Oct 17 15:44:20 2022
[ ]
step.eval_cam: Mon Oct 17 16:01:46 2022
{'iou': array([0.77959469, 0.38798271, 0.27075608, 0.43952468, 0.33498522,
       0.50049972, 0.68126723, 0.53167374, 0.57285536, 0.28491552,
       0.57017733, 0.44442232, 0.51781705, 0.53194815, 0.60693332,
       0.53190954, 0.44646047, 0.60181321, 0.46994949, 0.50663687,
       0.47367236]), 'miou': 0.4993235743484251}

Maybe you can traverse the parameter 'cam_eval_thres' to obtain the best score. This is a common process in most approaches.

[Zhengyang1995]

Okay， thank you so much for your help! It is really kind of you to reproduce it again. Thanks!