Bad results with other SUN-RGBD objects

[dnlwbr]

Hi, if I train on the same 10 classes from the sunrgbd dataset that are used in the paper, I get good results. But if I use 10 different classes from the same data set, the results are very bad. Actually not usable. Why is that? I have adapted type2class, type2onehotclass and type_mean_size in model_util_sunrgbd.py, as well as type2class in sunrgbd_utils.py and DEFAULT_TYPE_WHITELIST in sunrgbd_data.py. Have I forgotten something? My result looks like this:

Namespace(DUMP_DIR='eval_sunrgbd_custom10_v1', ap_iou_thresholds='0.25,0.5', batch_size=8, checkpoint_path='log_sunrgbd_custom10_v1/checkpoint.tar', cluster_sampling='seed_fps', conf_thresh=0.05, dataset='sunrgbd', dump_dir='eval_sunrgbd_custom10_v1', faster_eval=False, model='votenet', nms_iou=0.25, no_height=False, num_point=20000, num_target=256, per_class_proposal=True, shuffle_dataset=False, use_3d_nms=True, use_cls_nms=True, use_color=False, use_old_type_nms=False, use_sunrgbd_v2=False, vote_factor=1) Loaded checkpoint log_sunrgbd_custom10_v1/checkpoint.tar (epoch: 180) 2020-10-29 09:38:06.638843 eval mean box_loss: 0.345646 eval mean center_loss: 0.027933 eval mean heading_cls_loss: 0.871540 eval mean heading_reg_loss: 0.065775 eval mean loss: 5.452499 eval mean neg_ratio: 0.975985 eval mean obj_acc: 0.991017 eval mean objectness_loss: 0.021886 eval mean pos_ratio: 0.006967 eval mean sem_cls_loss: 0.852660 eval mean size_cls_loss: 0.852561 eval mean size_reg_loss: 0.079528 eval mean vote_loss: 0.103395 eval book Average Precision: 0.000103 eval bottle Average Precision: 0.000416 eval bowl Average Precision: 0.000405 eval cup Average Precision: 0.000541 eval keyboard Average Precision: 0.004161 eval laptop Average Precision: 0.001543 eval mouse Average Precision: 0.000022 eval paper Average Precision: 0.000192 eval plant Average Precision: 0.002528 eval telephone Average Precision: 0.001068 eval mAP: 0.001098 eval book Recall: 0.264706 eval bottle Recall: 0.180328 eval bowl Recall: 0.212500 eval cup Recall: 0.235294 eval keyboard Recall: 0.288557 eval laptop Recall: 0.247312 eval mouse Recall: 0.037037 eval paper Recall: 0.175182 eval plant Recall: 0.341463 eval telephone Recall: 0.291667 eval AR: 0.227405 eval book Average Precision: 0.000000 eval bottle Average Precision: 0.000000 eval bowl Average Precision: 0.000000 eval cup Average Precision: 0.000000 eval keyboard Average Precision: 0.000023 eval laptop Average Precision: 0.000002 eval mouse Average Precision: 0.000000 eval paper Average Precision: 0.000000 eval plant Average Precision: 0.000172 eval telephone Average Precision: 0.000000 eval mAP: 0.000020 eval book Recall: 0.014706 eval bottle Recall: 0.000000 eval bowl Recall: 0.000000 eval cup Recall: 0.000000 eval keyboard Recall: 0.009950 eval laptop Recall: 0.010753 eval mouse Recall: 0.000000 eval paper Recall: 0.007299 eval plant Recall: 0.073171 eval telephone Recall: 0.000000 eval AR: 0.011588

Thanks in advance!

[wisper181]

Hello,have you sovled this problem? I get the same problem yet, I want to train only one class(like box class in SUNRGBD),Where shuld I modify in all network?Wish your help,Thx 0.0

[dnlwbr]

Unfortunately not. My guess is that the model is very fine-tuned to the objects in the paper. Furthermore, objects like box or cup are probably more difficult than the ones in the paper because of their size. I will gladly be taught better, but at least I have not found a way.