AKEOPLUS-boris-bocquet
commented
3 months ago Hello. This is not easy to read what is inside your config py file. But I don't see the "metainfo" dictionnary. Indeed, even if you json coco file have these informations, you need to explicitely declare it into a metainfo dict that you will pass to your dataloader. Otherwise, mmdetection will use the default classes of the default coco datasets ('person', 'bicycle', 'car', 'motorcycle', etc.).
I had a similar issue (actually the one from this issue) and the reason was the missing metainfo dict.
So I recommand you to declare this metainfo dict, and pass it to dataloaders (I don't see any in your config). I invite you to base your config (at least concerning the dataloaders) on this part of the mmengine documentation. You may read the full mmengine chapter train with customized datasets
indices = np.concatenate(indices) File "<__array_function__ internals>", line 200, in concatenate ValueError: need at least one array to concatenate
Describe the bug 训练时报的错： LicenseRec) D:\studysoft\workplace\ORENEXT-main>python ./tools/train.py ./configs/ORENEXT/orenext_stonemlp_sparsefc_ptsdml.py D:\studysoft\Anaconda\envs\LicenseRec\lib\site-packages\mmcv__init__.py:20: UserWarning: On January 1, 2023, MMCV will release v2.0.0, in which it will remove co mponents related to the training process and add a data transformation module. In addition, it will rename the package names mmcv to mmcv-lite and mmcv-full to mmcv. See https://github.com/open-mmlab/mmcv/blob/master/docs/en/compatibility.md for more details. warnings.warn( fatal: not a git repository (or any of the parent directories): .git 2024-07-17 16:24:49,426 - mmdet - INFO - Environment info:
sys.platform: win32 Python: 3.8.19 (default, Mar 20 2024, 19:55:45) [MSC v.1916 64 bit (AMD64)] CUDA available: True GPU 0: NVIDIA GeForce RTX 4060 CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0 NVCC: Cuda compilation tools, release 10.0, V10.0.13 MSVC: 用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.40.33812 版 GCC: n/a PyTorch: 1.9.0+cu111 PyTorch compiling details: PyTorch built with:
TorchVision: 0.10.0+cu111 OpenCV: 4.10.0 MMCV: 1.7.2 MMCV Compiler: MSVC 192930148 MMCV CUDA Compiler: 11.1 MMDetection: 2.11.0+
2024-07-17 16:24:49,977 - mmdet - INFO - Distributed training: False 2024-07-17 16:24:50,564 - mmdet - INFO - Config: model = dict( type='PointRend', backbone=dict( type='ASMLP', embed_dim=64, depths=[2, 2, 2, 2], num_heads=[3, 6, 12, 24], shift_size=5, window_size=7, mlp_ratio=4.0, drop_rate=0.0, drop_path_rate=0.1, patch_norm=True, out_indices=(0, 1, 2, 3), use_checkpoint=False), neck=dict( type='FPNSPARSEFC', in_channels=[64, 128, 256, 512], out_channels=64, num_outs=5), rpn_head=dict( type='RPNHead', in_channels=64, feat_channels=64, anchor_generator=dict( type='AnchorGenerator', scales=[8], ratios=[0.5, 1.0, 2.0], strides=[4, 8, 16, 32, 64]), bbox_coder=dict( type='DeltaXYWHBBoxCoder', target_means=[0.0, 0.0, 0.0, 0.0], target_stds=[1.0, 1.0, 1.0, 1.0]), loss_cls=dict( type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0), loss_bbox=dict(type='L1Loss', loss_weight=1.0)), roi_head=dict( type='PointRendRoIHead', bbox_roi_extractor=dict( type='SingleRoIExtractor', roi_layer=dict(type='RoIAlign', output_size=7, sampling_ratio=0), out_channels=64, featmap_strides=[4, 8, 16, 32]), bbox_head=dict( type='Shared2FCBBoxHead', in_channels=64, fc_out_channels=64, roi_feat_size=7, num_classes=1, bbox_coder=dict( type='DeltaXYWHBBoxCoder', target_means=[0.0, 0.0, 0.0, 0.0], target_stds=[0.1, 0.1, 0.2, 0.2]), reg_class_agnostic=False, loss_cls=dict( type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0), loss_bbox=dict(type='PtsBorderLoss', loss_weight=0.5)), mask_roi_extractor=dict( type='GenericRoIExtractor', roi_layer=dict(type='SimpleRoIAlign', output_size=14), out_channels=64, featmap_strides=[4], aggregation='concat'), mask_head=dict( type='CoarseMaskHead', num_fcs=1, in_channels=64, conv_out_channels=64, fc_out_channels=64, num_classes=1, loss_mask=dict( type='CrossEntropyLoss', use_mask=True, loss_weight=0.1)), point_head=dict( type='MaskPointHead', num_fcs=1, in_channels=64, fc_channels=64, num_classes=1, coarse_pred_each_layer=True, loss_point=dict( type='CrossEntropyLoss1', use_mask=True, loss_weight=1.0, key_item_weight=0.5))), train_cfg=dict( rpn=dict( assigner=dict( type='MaxIoUAssigner', pos_iou_thr=0.7, neg_iou_thr=0.3, min_pos_iou=0.3, match_low_quality=True, ignore_iof_thr=-1), sampler=dict( type='RandomSampler', num=256, pos_fraction=0.5, neg_pos_ub=-1, add_gt_as_proposals=False), allowed_border=-1, pos_weight=-1, debug=False), rpn_proposal=dict( nms_pre=2000, max_per_img=1000, nms=dict(type='nms', iou_threshold=0.7), min_bbox_size=0), rcnn=dict( assigner=dict( type='MaxIoUAssigner', pos_iou_thr=0.5, neg_iou_thr=0.5, min_pos_iou=0.5, match_low_quality=True, ignore_iof_thr=-1), sampler=dict( type='RandomSampler', num=512, pos_fraction=0.25, neg_pos_ub=-1, add_gt_as_proposals=True), mask_size=7, pos_weight=-1, debug=False, num_points=196, oversample_ratio=3, importance_sample_ratio=0.75)), test_cfg=dict( rpn=dict( nms_pre=1000, max_per_img=1000, nms=dict(type='nms', iou_threshold=0.7), min_bbox_size=0), rcnn=dict( score_thr=0.05, nms=dict(type='nms', iou_threshold=0.5), max_per_img=100, mask_thr_binary=0.5, subdivision_steps=5, subdivision_num_points=784, scale_factor=2))) dataset_type = 'CocoDataset' data_root = 'data/coco/stones/' img_norm_cfg = dict( mean=[103.53, 116.28, 123.675], std=[1.0, 1.0, 1.0], to_rgb=False) train_pipeline = [ dict(type='LoadImageFromFile'), dict(type='LoadAnnotations', with_bbox=True, with_mask=True), dict( type='Resize', img_scale=[(320, 160), (320, 192), (320, 224), (320, 256), (320, 288), (320, 320)], multiscale_mode='value', keep_ratio=True), dict(type='RandomFlip', flip_ratio=0.5), dict( type='Normalize', mean=[103.53, 116.28, 123.675], std=[1.0, 1.0, 1.0], to_rgb=False), dict(type='Pad', size_divisor=32), dict(type='DefaultFormatBundle'), dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks']) ] test_pipeline = [ dict(type='LoadImageFromFile'), dict( type='MultiScaleFlipAug', img_scale=(320, 320), flip=False, transforms=[ dict(type='Resize', keep_ratio=True), dict(type='RandomFlip'), dict( type='Normalize', mean=[103.53, 116.28, 123.675], std=[1.0, 1.0, 1.0], to_rgb=False), dict(type='Pad', size_divisor=32), dict(type='ImageToTensor', keys=['img']), dict(type='Collect', keys=['img']) ]) ] data = dict( samples_per_gpu=8, workers_per_gpu=2, train=dict( type='CocoDataset', ann_file='data/coco/stones/annotations/instances_train2017.json', img_prefix='data/coco/stones/train2017/', pipeline=[ dict(type='LoadImageFromFile'), dict(type='LoadAnnotations', with_bbox=True, with_mask=True), dict( type='Resize', img_scale=[(320, 160), (320, 192), (320, 224), (320, 256), (320, 288), (320, 320)], multiscale_mode='value', keep_ratio=True), dict(type='RandomFlip', flip_ratio=0.5), dict( type='Normalize', mean=[103.53, 116.28, 123.675], std=[1.0, 1.0, 1.0], to_rgb=False), dict(type='Pad', size_divisor=32), dict(type='DefaultFormatBundle'), dict( type='Collect', keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks']) ]), val=dict( type='CocoDataset', ann_file='data/coco/stones/annotations/instances_val2017.json', img_prefix='data/coco/stones/val2017/', pipeline=[ dict(type='LoadImageFromFile'), dict( type='MultiScaleFlipAug', img_scale=(320, 320), flip=False, transforms=[ dict(type='Resize', keep_ratio=True), dict(type='RandomFlip'), dict( type='Normalize', mean=[103.53, 116.28, 123.675], std=[1.0, 1.0, 1.0], to_rgb=False), dict(type='Pad', size_divisor=32), dict(type='ImageToTensor', keys=['img']), dict(type='Collect', keys=['img']) ]) ]), test=dict( type='CocoDataset', ann_file='data/coco/stones/annotations/instances_val2017.json', img_prefix='data/coco/stones/val2017/', pipeline=[ dict(type='LoadImageFromFile'), dict( type='MultiScaleFlipAug', img_scale=(320, 320), flip=False, transforms=[ dict(type='Resize', keep_ratio=True), dict(type='RandomFlip'), dict( type='Normalize', mean=[103.53, 116.28, 123.675], std=[1.0, 1.0, 1.0], to_rgb=False), dict(type='Pad', size_divisor=32), dict(type='ImageToTensor', keys=['img']), dict(type='Collect', keys=['img']) ]) ])) evaluation = dict(metric=['bbox', 'segm']) optimizer = dict(type='SGD', lr=0.02, momentum=0.9, weight_decay=0.0001) optimizer_config = dict(grad_clip=None) lr_config = dict( policy='step', warmup='linear', warmup_iters=500, warmup_ratio=0.001, step=[8, 11]) runner = dict(type='EpochBasedRunner', max_epochs=12) checkpoint_config = dict(interval=1) log_config = dict(interval=50, hooks=[dict(type='TextLoggerHook')]) custom_hooks = [dict(type='NumClassCheckHook')] dist_params = dict(backend='nccl') log_level = 'INFO' load_from = None resume_from = None workflow = [('train', 1)] work_dir = './work_dirs/orenext_stonemlp_sparsefc_ptsdml1.py' gpu_ids = range(0, 1)
loading annotations into memory... Done (t=0.00s) creating index... index created! fatal: not a git repository (or any of the parent directories): .git loading annotations into memory... Done (t=0.33s) creating index... index created! 2024-07-17 16:24:51,932 - mmdet - INFO - Start running, host: xiaoran@小冉, work_dir: D:\studysoft\workplace\ORENEXT-main\work_dirs\orenext_stonemlp_sparsefc_ptsdml1.py 2024-07-17 16:24:51,932 - mmdet - INFO - Hooks will be executed in the following order: before_run: (VERY_HIGH ) StepLrUpdaterHook (NORMAL ) CheckpointHook (NORMAL ) EvalHook (VERY_LOW ) TextLoggerHook
before_train_epoch: (VERY_HIGH ) StepLrUpdaterHook (NORMAL ) EvalHook (NORMAL ) NumClassCheckHook (LOW ) IterTimerHook (VERY_LOW ) TextLoggerHook
before_train_iter: (VERY_HIGH ) StepLrUpdaterHook (LOW ) IterTimerHook
after_train_iter: (ABOVE_NORMAL) OptimizerHook (NORMAL ) CheckpointHook (NORMAL ) EvalHook (LOW ) IterTimerHook (VERY_LOW ) TextLoggerHook
after_train_epoch: (NORMAL ) CheckpointHook (NORMAL ) EvalHook (VERY_LOW ) TextLoggerHook
before_val_epoch: (NORMAL ) NumClassCheckHook (LOW ) IterTimerHook (VERY_LOW ) TextLoggerHook
before_val_iter: (LOW ) IterTimerHook
after_val_iter: (LOW ) IterTimerHook
after_val_epoch: (VERY_LOW ) TextLoggerHook
after_run: (VERY_LOW ) TextLoggerHook
2024-07-17 16:24:51,933 - mmdet - INFO - workflow: [('train', 1)], max: 12 epochs 2024-07-17 16:24:51,933 - mmdet - INFO - Checkpoints will be saved to D:\studysoft\workplace\ORENEXT-main\work_dirs\orenext_stonemlp_sparsefc_ptsdml1.py by HardDiskBackend. D:\studysoft\Anaconda\envs\LicenseRec\lib\site-packages\mmcv__init.py:20: UserWarning: On January 1, 2023, MMCV will release v2.0.0, in which it will remove co mponents related to the training process and add a data transformation module. In addition, it will rename the package names mmcv to mmcv-lite and mmcv-full to mmcv. See https://github.com/open-mmlab/mmcv/blob/master/docs/en/compatibility.md for more details. warnings.warn( D:\studysoft\Anaconda\envs\LicenseRec\lib\site-packages\mmcv__init__.py:20: UserWarning: On January 1, 2023, MMCV will release v2.0.0, in which it will remove co mponents related to the training process and add a data transformation module. In addition, it will rename the package names mmcv to mmcv-lite and mmcv-full to mmcv. See https://github.com/open-mmlab/mmcv/blob/master/docs/en/compatibility.md for more details. warnings.warn( Indices before concatenation: [] Traceback (most recent call last): File "./tools/train.py", line 192, in
main()
File "./tools/train.py", line 181, in main
train_detector(
File "D:\studysoft\Anaconda\envs\LicenseRec\lib\site-packages\mmdet-2.11.0-py3.8.egg\mmdet\apis\train.py", line 185, in train_detector
runner.run(data_loaders, cfg.workflow)
File "D:\studysoft\Anaconda\envs\LicenseRec\lib\site-packages\mmcv\runner\epoch_based_runner.py", line 136, in run
epoch_runner(data_loaders[i], **kwargs)
File "D:\studysoft\Anaconda\envs\LicenseRec\lib\site-packages\mmcv\runner\epoch_based_runner.py", line 49, in train
for i, data_batch in enumerate(self.data_loader):
File "D:\studysoft\Anaconda\envs\LicenseRec\lib\site-packages\torch\utils\data\dataloader.py", line 359, in iter
return self._get_iterator()
File "D:\studysoft\Anaconda\envs\LicenseRec\lib\site-packages\torch\utils\data\dataloader.py", line 305, in _get_iterator
return _MultiProcessingDataLoaderIter(self)
File "D:\studysoft\Anaconda\envs\LicenseRec\lib\site-packages\torch\utils\data\dataloader.py", line 512, in _next_index
return next(self._sampler_iter) # may raise StopIteration
File "D:\studysoft\Anaconda\envs\LicenseRec\lib\site-packages\torch\utils\data\sampler.py", line 226, in iter
for idx in self.sampler:
File "D:\studysoft\Anaconda\envs\LicenseRec\lib\site-packages\mmdet-2.11.0-py3.8.egg\mmdet\datasets\samplers\group_sampler.py", line 37, in iter
indices = np.concatenate(indices)
File "<array_function__ internals>", line 200, in concatenate
ValueError: need at least one array to concatenate
运行参数：python ./tools/train.py ./configs/ORENEXT/orenext_stonemlp_sparsefc_ptsdml.py
pretrained='./pretrained/asmlp_nano_patch4_shift5_224.pth',
backbone=dict( type='ASMLP', embed_dim=64, depths=[2, 2, 2, 2], num_heads=[3, 6, 12, 24], shift_size=5, window_size=7, mlp_ratio=4.0, drop_rate=0.0, drop_path_rate=0.1, patch_norm=True, out_indices=(0, 1, 2, 3), use_checkpoint=False), neck=dict( type='FPNSPARSEFC', in_channels=[64, 128, 256, 512], out_channels=64, num_outs=5), rpn_head=dict( type='RPNHead', in_channels=64, feat_channels=64, anchor_generator=dict( type='AnchorGenerator', scales=[8], ratios=[0.5, 1.0, 2.0], strides=[4, 8, 16, 32, 64]), bbox_coder=dict( type='DeltaXYWHBBoxCoder', target_means=[0.0, 0.0, 0.0, 0.0], target_stds=[1.0, 1.0, 1.0, 1.0]), loss_cls=dict( type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0), loss_bbox=dict(type='L1Loss', loss_weight=1.0)), roi_head=dict( type='PointRendRoIHead', bbox_roi_extractor=dict( type='SingleRoIExtractor', roi_layer=dict(type='RoIAlign', output_size=7, sampling_ratio=0), out_channels=64, featmap_strides=[4, 8, 16, 32]), bbox_head=dict( type='Shared2FCBBoxHead', in_channels=64, fc_out_channels=64, roi_feat_size=7, num_classes=1, bbox_coder=dict( type='DeltaXYWHBBoxCoder', target_means=[0.0, 0.0, 0.0, 0.0], target_stds=[0.1, 0.1, 0.2, 0.2]), reg_class_agnostic=False, loss_cls=dict( type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0), loss_bbox=dict(type='PtsBorderLoss', loss_weight=0.5)), mask_roi_extractor=dict( type='GenericRoIExtractor', roi_layer=dict(type='SimpleRoIAlign', output_size=14), out_channels=64, featmap_strides=[4], aggregation='concat'), mask_head=dict( type='CoarseMaskHead', num_fcs=1, in_channels=64, conv_out_channels=64, fc_out_channels=64, num_classes=1, loss_mask=dict( type='CrossEntropyLoss', use_mask=True, loss_weight=0.1)), point_head=dict( type='MaskPointHead', num_fcs=1, in_channels=64, fc_channels=64, num_classes=1, coarse_pred_each_layer=True, loss_point=dict(type='CrossEntropyLoss1', use_mask=True, loss_weight=1.0,key_item_weight=0.5))), train_cfg=dict( rpn=dict( assigner=dict( type='MaxIoUAssigner', pos_iou_thr=0.7, neg_iou_thr=0.3, min_pos_iou=0.3, match_low_quality=True, ignore_iof_thr=-1), sampler=dict( type='RandomSampler', num=256, pos_fraction=0.5, neg_pos_ub=-1, add_gt_as_proposals=False), allowed_border=-1, pos_weight=-1, debug=False), rpn_proposal=dict( nms_pre=2000, max_per_img=1000, nms=dict(type='nms', iou_threshold=0.7), min_bbox_size=0), rcnn=dict( assigner=dict( type='MaxIoUAssigner', pos_iou_thr=0.5, neg_iou_thr=0.5, min_pos_iou=0.5, match_low_quality=True, ignore_iof_thr=-1), sampler=dict( type='RandomSampler', num=512, pos_fraction=0.25, neg_pos_ub=-1, add_gt_as_proposals=True), mask_size=7, pos_weight=-1, debug=False, num_points=196, oversample_ratio=3, importance_sample_ratio=0.75)), test_cfg=dict( rpn=dict( nms_pre=1000, max_per_img=1000, nms=dict(type='nms', iou_threshold=0.7), min_bbox_size=0), rcnn=dict( score_thr=0.05, nms=dict(type='nms', iou_threshold=0.5), max_per_img=100, mask_thr_binary=0.5, subdivision_steps=5, subdivision_num_points=784, scale_factor=2))) dataset_type = 'CocoDataset' data_root = 'data/coco/stones/' img_norm_cfg = dict( mean=[103.53, 116.28, 123.675], std=[1.0, 1.0, 1.0], to_rgb=False) train_pipeline = [ dict(type='LoadImageFromFile'), dict(type='LoadAnnotations', with_bbox=True, with_mask=True), dict( type='Resize', img_scale=[(320, 160), (320, 192), (320, 224), (320, 256), (320, 288), (320, 320)], multiscale_mode='value', keep_ratio=True), dict(type='RandomFlip', flip_ratio=0.5), dict( type='Normalize', mean=[103.53, 116.28, 123.675], std=[1.0, 1.0, 1.0], to_rgb=False), dict(type='Pad', size_divisor=32), dict(type='DefaultFormatBundle'), dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks']) ] test_pipeline = [ dict(type='LoadImageFromFile'), dict( type='MultiScaleFlipAug', img_scale=(320, 320), flip=False, transforms=[ dict(type='Resize', keep_ratio=True), dict(type='RandomFlip'), dict( type='Normalize', mean=[103.53, 116.28, 123.675], std=[1.0, 1.0, 1.0], to_rgb=False), dict(type='Pad', size_divisor=32), dict(type='ImageToTensor', keys=['img']), dict(type='Collect', keys=['img']) ]) ] data = dict( samples_per_gpu=8, workers_per_gpu=2, train=dict( type='CocoDataset', ann_file='data/coco/stones/annotations/instances_train2017.json', img_prefix='data/coco/stones/train2017/', pipeline=[ dict(type='LoadImageFromFile'), dict(type='LoadAnnotations', with_bbox=True, with_mask=True), dict( type='Resize', img_scale=[(320, 160), (320, 192), (320, 224), (320, 256), (320, 288), (320, 320)], multiscale_mode='value', keep_ratio=True), dict(type='RandomFlip', flip_ratio=0.5), dict( type='Normalize', mean=[103.53, 116.28, 123.675], std=[1.0, 1.0, 1.0], to_rgb=False), dict(type='Pad', size_divisor=32), dict(type='DefaultFormatBundle'), dict( type='Collect', keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks']) ]), val=dict( type='CocoDataset', ann_file='data/coco/stones/annotations/instances_val2017.json', img_prefix='data/coco/stones/val2017/', pipeline=[ dict(type='LoadImageFromFile'), dict( type='MultiScaleFlipAug', img_scale=(320, 320), flip=False, transforms=[ dict(type='Resize', keep_ratio=True), dict(type='RandomFlip'), dict( type='Normalize', mean=[103.53, 116.28, 123.675], std=[1.0, 1.0, 1.0], to_rgb=False), dict(type='Pad', size_divisor=32), dict(type='ImageToTensor', keys=['img']), dict(type='Collect', keys=['img']) ]) ]), test=dict( type='CocoDataset', ann_file='data/coco/stones/annotations/instances_val2017.json', img_prefix='data/coco/stones/val2017/', pipeline=[ dict(type='LoadImageFromFile'), dict( type='MultiScaleFlipAug', img_scale=(320, 320), flip=False, transforms=[ dict(type='Resize', keep_ratio=True), dict(type='RandomFlip'), dict( type='Normalize', mean=[103.53, 116.28, 123.675], std=[1.0, 1.0, 1.0], to_rgb=False), dict(type='Pad', size_divisor=32), dict(type='ImageToTensor', keys=['img']), dict(type='Collect', keys=['img']) ]) ])) evaluation = dict(metric=['bbox', 'segm']) optimizer = dict(type='SGD', lr=0.02, momentum=0.9, weight_decay=0.0001) optimizer_config = dict(grad_clip=None) lr_config = dict( policy='step', warmup='linear', warmup_iters=500, warmup_ratio=0.001, step=[8, 11]) runner = dict(type='EpochBasedRunner', max_epochs=12) checkpoint_config = dict(interval=1) log_config = dict(interval=50, hooks=[dict(type='TextLoggerHook')]) custom_hooks = [dict(type='NumClassCheckHook')] dist_params = dict(backend='nccl') log_level = 'INFO' load_from = None resume_from = None workflow = [('train', 1)] work_dir = './work_dirs/orenext_stonemlp_sparsefc_ptsdml1.py' gpu_ids = range(0, 1)
公开数据集COCO
{ "images": [ { "height": 300, "width": 300, "id": 1, "file_name": "00000.png" }, { "height": 300, "width": 300, "id": 2, "file_name": "00001.png" }, { "height": 300, "width": 300, "id": 3, "file_name": "00002.png" }, { "height": 300, "width": 300, "id": 4, "file_name": "00003.png" }, { "height": 300, "width": 300, "id": 5, "file_name": "00004.png" }