search

lyclyc52 / NeRF_RPN

[CVPR2023] NeRF-RPN: A general framework for object detection in NeRFs
MIT License
221 stars 12 forks source link

a RuntimeError #10

Open lgwsgc opened 1 year ago

lgwsgc commented 1 year ago

Hello, I am a white, after I built the deep learning environment, put front3d_rpn_data into the data folder, according to the train.sh configuration parameter, "RuntimeError: split_with_sizes expects split_sizes to sum exactly to 6 (input tensor's size at dimension -1), but got split_sizes=[3, 1, 1, 1, 1] ", can you help explain

lyclyc52 commented 1 year ago

Could you provide more details e.g. your training bash, full error description?

zymk9 commented 1 year ago

That is likely due to a dimension mismatch between the input bounding boxes and the expected bounding box format.

When training a model using OBB, please specify --rotated_bbox in the training args and use the data under obb folders. Here it seems AABB data are supplied while the model expects OBB?

lgwsgc commented 1 year ago

Thank you very much for your reply, which will provide me with great help. In addition, the gpu I used was 4060-8g in Windows, and the pytorch version was 2.0.1+cu118, because when batch_size=1, the memory still overran, so I only kept 50 samples of the front3d dataset (originally 159), may I ask the helpful Contributor, what was the hardware condition you used at that time? Didn't seem to notice it in the article. The following is my train.sh and full error description. Additional, would you have any suggestions for learning in nerf direction for me? train.sh： parser.add_argument('--mode', default='train', choices=['train', 'eval', 'benchmark']) parser.add_argument('--dataset_name', '-dn', default='front3d', choices=['hypersim', 'front3d', 'general', 'scannet'])

parser.add_argument('--features_path',
                    default=r'D:\Code\Literature\2restration\NeRF_RPN-main\data\front3d_rpn_data\features',
                    help='Path to the features.')
parser.add_argument('--boxes_path',
                    default=r'D:\Code\Literature\2restration\NeRF_RPN-main\data\front3d_rpn_data\obb',
                    help='Path to the gt boxes.')
parser.add_argument('--save_path',
                    default=r'D:\Code\Literature\2restration\NeRF_RPN-main\results\front3d_anchor_swins',
                    help='Path to save the model.')
parser.add_argument('--dataset_split',
                    default=r'D:\Code\Literature\2restration\NeRF_RPN-main\data\front3d_rpn_data\3dfront_split.npz',
                    help='Path to the dataset split file.')
parser.add_argument('--preload', action='store_true', help='Preload the features and boxes.')

parser.add_argument('--checkpoint',
                    default='',
                    help='Path to the checkpoint to load.')
parser.add_argument('--load_backbone_only', action='store_true', help='Only load the backbone.')
parser.add_argument('--backbone_type', type=str, default='swin_s',
                    choices=['resnet', 'vgg_AF', 'vgg_EF',  'swin_t', 'swin_s', 'swin_b', 'swin_l'],)
parser.add_argument('--freeze_backbone', action='store_true', help='Freeze the backbone.')

full error description： D:\Project_entertainment\python\python.exe D:\Code\Literature\2restration\NeRF_RPN-main\nerf_rpn\run_rpn.py Ping �����Ҳ������� ff10554454651564539514c54593051306844546c424c4f314a6c5a47567459.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� f10157356a6554747a62334a304c585a6c636e52705932567a587a45754d44.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� f10274454f6c78446232526c584578706447567959585231636d56634d6e4a.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� f1036c633352795958527062323563546d5653526c39535545347462574670.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� f104626c78755a584a6d58334a7762673d3d.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� [2023-07-01 16:59:51,115 INFO] Constructing model... [2023-07-01 16:59:51,775 INFO] Loading dataset split from D:\Code\Literature\2restration\NeRF_RPN-main\data\front3d_rpn_data\3dfront_split.npz. 100%|██████████| 40/40 [00:00<00:00, 8027.38it/s] 100%|██████████| 5/5 [00:00<?, ?it/s] [2023-07-01 16:59:51,785 INFO] Loaded 40 training scenes, 5 validation scenes Ping �����Ҳ������� f830554454651564539514c54593051306844546c424c4f314a6c5a47567459.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� f6a0554454651564539514c54593051306844546c424c4f314a6c5a47567459.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� 830157356a6554747a62334a304c585a6c636e52705932567a587a45754d44.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� fd50554454651564539514c54593051306844546c424c4f314a6c5a47567459.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� f190554454651564539514c54593051306844546c424c4f314a6c5a47567459.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� 190157356a6554747a62334a304c585a6c636e52705932567a587a45754d44.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� 6a0157356a6554747a62334a304c585a6c636e52705932567a587a45754d44.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� 830274454f6c78446232526c584578706447567959585231636d56634d6e4a.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� 83036c633352795958527062323563546d5653526c39535545347462574670.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� d50157356a6554747a62334a304c585a6c636e52705932567a587a45754d44.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� 6a0274454f6c78446232526c584578706447567959585231636d56634d6e4a.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� d50274454f6c78446232526c584578706447567959585231636d56634d6e4a.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� 190274454f6c78446232526c584578706447567959585231636d56634d6e4a.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� 8304626c78755a584a6d58334a7762673d3d.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� d5036c633352795958527062323563546d5653526c39535545347462574670.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� 19036c633352795958527062323563546d5653526c39535545347462574670.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� d504626c78755a584a6d58334a7762673d3d.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� 6a036c633352795958527062323563546d5653526c39535545347462574670.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� 1904626c78755a584a6d58334a7762673d3d.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Ping �����Ҳ������� 6a04626c78755a584a6d58334a7762673d3d.sub.deliverycontent.online����������ƣ�Ȼ�����ԡ� Traceback (most recent call last): File "D:\Code\Literature\2restration\NeRF_RPN-main\nerf_rpn\run_rpn.py", line 705, in main() File "D:\Code\Literature\2restration\NeRF_RPN-main\nerf_rpn\run_rpn.py", line 692, in main trainer.train_loop() File "D:\Code\Literature\2restration\NeRF_RPN-main\nerf_rpn\run_rpn.py", line 368, in train_loop self.train_epoch(epoch) File "D:\Code\Literature\2restration\NeRF_RPN-main\nerf_rpn\run_rpn.py", line 394, in train_epoch proposals, losses, scores = self.model(rgbsigma, boxes) File "C:\Users\Redemancy\AppData\Roaming\Python\Python310\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(*args, *kwargs) File "D:\Code\Literature\2restration\NeRF_RPN-main\nerf_rpn\model\nerf_rpn.py", line 213, in forward proposals, level_index, proposal_losses, scores = self.rpn( File "C:\Users\Redemancy\AppData\Roaming\Python\Python310\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl return forward_call(args, **kwargs) File "D:\Code\Literature\2restration\NeRF_RPN-main\nerf_rpn\model\rpn.py", line 526, in forward loss_objectness, loss_rpn_box_reg, loss_rpn_box_reg_2d = self.compute_loss( File "D:\Code\Literature\2restration\NeRF_RPN-main\nerf_rpn\model\rpn.py", line 436, in compute_loss pred_bbox = obb2points_3d(pred_bbox[sampled_pos_inds]) File "D:\Code\Literature\2restration\NeRF_RPN-main\nerf_rpn\model\coder\misc.py", line 98, in obb2points_3d center, w, l, h, theta = torch.split(obboxes, [3, 1, 1, 1, 1], dim=-1) File "C:\Users\Redemancy\AppData\Roaming\Python\Python310\site-packages\torch\functional.py", line 189, in split return tensor.split(split_size_or_sections, dim) File "C:\Users\Redemancy\AppData\Roaming\Python\Python310\site-packages\torch_tensor.py", line 803, in split return torch._VF.split_with_sizes(self, split_size, dim) RuntimeError: split_with_sizes expects split_sizes to sum exactly to 6 (input tensor's size at dimension -1), but got split_sizes=[3, 1, 1, 1, 1]

lgwsgc commented 1 year ago

That is likely due to a dimension mismatch between the input bounding boxes and the expected bounding box format.

When training a model using OBB, please specify --rotated_bbox in the training args and use the data under obb folders. Here it seems AABB data are supplied while the model expects OBB?

Thank you very much for your answer. At present, due to my poor code base, I am not able to modify the code according to specific conditions, but I will gradually learn, thank you