detect four class

[DMing11]

Hi @happinesslz I want to change your code to detect four class ,but got some errors, I changed : cfgs/LI_Fusion_with_attention_use_ce_loss.yaml, line1: CLASSES: Multi_class kitti_rcnn_dataset.py , line45 add :
if classes == 'Multi_class': self.classes = ('Background', 'Car', 'Pedestrian', 'Cyclist', 'Van') aug_scene_root_dir = os.path.join(root_dir, 'KITTI', 'aug_scene_multi_class')

got such error: File "/home/data/lidar_camera_fusion/EPNet/tools/train_rcnn.py", line 279, in lr_scheduler_each_iter = (cfg.TRAIN.OPTIMIZER == 'adam_onecycle') File "/home/data/lidar_camera_fusion/EPNet/tools/../tools/train_utils/train_utils.py", line 197, in train loss, tb_dict, disp_dict = self._train_it(batch) File "/home/data/lidar_camera_fusion/EPNet/tools/../tools/train_utils/train_utils.py", line 130, in _train_it loss, tb_dict, disp_dict = self.model_fn(self.model, batch) File "/home/data/lidar_camera_fusion/EPNet/tools/../lib/net/train_functions.py", line 81, in model_fn rcnn_loss = get_rcnn_loss(model, ret_dict, tb_dict) File "/home/data/lidar_camera_fusion/EPNet/tools/../lib/net/train_functions.py", line 204, in get_rcnn_loss batch_loss_cls = F.binary_cross_entropy(torch.sigmoid(rcnn_cls_flat), cls_label, reduction='none') File "/home/dming/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/functional.py", line 2070, in binary_cross_entropy "!= input nelement ({})".format(target.numel(), input.numel())) ValueError: Target and input must have the same number of elements. target nelement (64) != input nelement (320)

Have you done this job that detect four class ? or give me some advice. Many Thanks !!!

[happinesslz]

@DMing11 Maybe you need delete the class "'Van'" in the config file for three class instead of four class, because both "Car" and "Van" are regarded as the only "Car" class in the data processing. If you want train four class, please modify this code in filtrate_objects

[DMing11]

@happinesslz thank for your answer I delete the class ''van'' but got another error: ValueError: Target and input must have the same number of elements. target nelement (64) != input nelement (256)

[DMing11]

@happinesslz I think it's a problem with LOSS https://github.com/happinesslz/EPNet/blob/0123c341243846aa3b412addcb9e2c07fd305237/lib/net/train_functions.py#L202 After adding class, it is no longer a two-class classification and should not be used again BinaryCrossEntropy？ but I don't know how to modify, can you give me some suggestion?

[happinesslz]

@DMing11 Please change the loss of RCNN stage (BinaryCrossEntropy-->CrossEntropy) in the config file. Maybe you also need modify the code which is related with the classification branch on RCNN stage. In addition, For three class, you could refer Openpcdet.

[DMing11]

@happinesslz Many thanks!! I will refer Openpcdet, In addition, I find https://github.com/happinesslz/EPNet/blob/0123c341243846aa3b412addcb9e2c07fd305237/lib/net/train_functions.py#L208 maybe should change "cfg.TRAIN.LOSS_CLS " --> "cfg.RCNN.LOSS_CLS"before change the loss of RCNN stage (BinaryCrossEntropy-->CrossEntropy) in the config file?

Because when I modify BinaryCrossEntropy to CrossEntropy in the config file, got an error: File "/home/data/lidar_camera_fusion/EPNet/tools/../lib/net/train_functions.py", line 208, in get_rcnn_loss elif cfg.TRAIN.LOSS_CLS == 'CrossEntropy': AttributeError: 'EasyDict' object has no attribute 'LOSS_CLS'

[happinesslz]

@DMing11 I mean "RCNN.LOSS_CLS" not "TRAIN.LOSS_CLS" config. There is no defination of "TRAIN.LOSS_CLS" in the config file.

[DMing11]

@happinesslz Maybe I didn’t express it clearly, I have modified BinaryCrossEntropy to CrossEntropy in https://github.com/happinesslz/EPNet/blob/0123c341243846aa3b412addcb9e2c07fd305237/tools/cfgs/LI_Fusion_with_attention_use_ce_loss.yaml#L127 , but got an error: File "/home/data/lidar_camera_fusion/EPNet/tools/../lib/net/train_functions.py", line 208, in get_rcnn_loss elif cfg.TRAIN.LOSS_CLS == 'CrossEntropy': AttributeError: 'EasyDict' object has no attribute 'LOSS_CLS', beacuse there is only " if cfg.RCNN.LOSS_CLS == 'SigmoidFocalLoss':"and "elif cfg.RCNN.LOSS_CLS == 'BinaryCrossEntropy':"in https://github.com/happinesslz/EPNet/blob/0123c341243846aa3b412addcb9e2c07fd305237/lib/net/train_functions.py#L202, no "cfg.RCNN.LOSS_CLS == 'CrossEntropy'"

So I guess the code maybe wrong?, "cfg.TRAIN.LOSS_CLS " should be "cfg.RCNN.LOSS_CLS" in https://github.com/happinesslz/EPNet/blob/0123c341243846aa3b412addcb9e2c07fd305237/lib/net/train_functions.py#L208

[happinesslz]

Yes, you are right. "cfg.TRAIN.LOSS_CLS " should be "cfg.RCNN.LOSS_CLS" in the train_functions.py for training multi classes.

[DMing11]

@happinesslz thank you ! I modify cfg.TRAIN.LOSS_CLS to cfg.RCNN.LOSS_CLS in the https://github.com/happinesslz/EPNet/blob/0123c341243846aa3b412addcb9e2c07fd305237/lib/net/train_functions.py#L208, got an error: File "/home/data/lidar_camera_fusion/EPNet/tools/../lib/net/train_functions.py", line 212, in get_rcnn_loss batch_loss_cls = cls_loss_func(rcnn_cls_reshape, cls_target) File "/home/dming/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call result = self.forward(*input, **kwargs) File "/home/dming/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/modules/loss.py", line 916, in forward ignore_index=self.ignore_index, reduction=self.reduction) File "/home/dming/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/functional.py", line 2021, in cross_entropy return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction) File "/home/dming/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/nn/functional.py", line 1838, in nll_loss ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index) RuntimeError: weight tensor should be defined either for all 5 classes or no classes but got weight tensor of shape: [3] at /opt/conda/conda-bld/pytorch_1579022060824/work/aten/src/THCUNN/generic/ClassNLLCriterion.cu:42

I modify https://github.com/happinesslz/EPNet/blob/0123c341243846aa3b412addcb9e2c07fd305237/tools/cfgs/LI_Fusion_with_attention_use_ce_loss.yaml#L130 to "CLS_WEIGHT: [1.0, 1.0, 1.0, 1.0, 1.0]", got another error :

File "/home/data/lidar_camera_fusion/EPNet/tools/../lib/net/train_functions.py", line 81, in model_fn rcnn_loss = get_rcnn_loss(model, ret_dict, tb_dict) File "/home/data/lidar_camera_fusion/EPNet/tools/../lib/net/train_functions.py", line 215, in get_rcnn_loss rcnn_loss_cls = (batch_loss_cls.mean(dim=1) * cls_valid_mask).sum() / normalizer IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

I modify https://github.com/happinesslz/EPNet/blob/0123c341243846aa3b412addcb9e2c07fd305237/lib/net/train_functions.py#L215 to "*rcnn_loss_cls = (batch_loss_cls cls_valid_mask).sum() / normalizer**" , got another error :

UnboundLocalError: local variable 'rcnn_cls_flat' referenced before assignment

I modify https://github.com/happinesslz/EPNet/blob/0123c341243846aa3b412addcb9e2c07fd305237/lib/net/train_functions.py#L236 "rcnn_cls_flat" to "rcnn_cls_reshape" , when run epochs: 0 train: 6%|▌ | 205/3687 got another error :

File "/home/data/lidar_camera_fusion/EPNet/tools/train_rcnn.py", line 279, in lr_scheduler_each_iter = (cfg.TRAIN.OPTIMIZER == 'adam_onecycle') File "/home/data/lidar_camera_fusion/EPNet/tools/../tools/train_utils/train_utils.py", line 197, in train loss, tb_dict, disp_dict = self._train_it(batch) File "/home/data/lidar_camera_fusion/EPNet/tools/../tools/train_utils/train_utils.py", line 130, in _train_it loss, tb_dict, disp_dict = self.model_fn(self.model, batch) File "/home/data/lidar_camera_fusion/EPNet/tools/../lib/net/train_functions.py", line 81, in model_fn rcnn_loss = get_rcnn_loss(model, ret_dict, tb_dict) File "/home/data/lidar_camera_fusion/EPNet/tools/../lib/net/train_functions.py", line 250, in get_rcnn_loss iou_branch_pred = iou_branch_pred_fg_mask) File "/home/data/lidar_camera_fusion/EPNet/tools/../lib/utils/loss_utils.py", line 332, in get_reg_loss iou_tmp = cls_score * iou_tmp RuntimeError: The size of tensor a (5) must match the size of tensor b (3) at non-singleton dimension 1

I don't know what the error means? can you give me some suggestion? I'm sorry to have taken so much of your time, many thinks!

[happinesslz]

@DMing11 Please provide your e-mail to me. Last year, I have tried to convert the single-class to multi-class on PointRCNN. Maybe it will be helpful you achieve the multi-class on EPNet, which is based on PointRCNN.

[DMing11]

liming_zhang_1@163.com @happinesslz thank you very much!

[KevinTang29]

Hello @happinesslz , I also want to modify your code to detect multiple classes and noticed this issue. Is there any progress now?

[TimGor1997]

@DMing11 Please provide your e-mail to me. Last year, I have tried to convert the single-class to multi-class on PointRCNN. Maybe it will be helpful you achieve the multi-class on EPNet, which is based on PointRCNN.

Dear @happinesslz , I want to achieve the multi-class on EPNet too, could you please share you work on PointRCNN wicth convert single-class to multi-class? My email address is 595603009@qq.com Thank you very much!