search

lyuwenyu / RT-DETR

[CVPR 2024] Official RT-DETR (RTDETR paddle pytorch), Real-Time DEtection TRansformer, DETRs Beat YOLOs on Real-time Object Detection. 🔥 🔥 🔥
Apache License 2.0
2.61k stars 303 forks source link

Error in finetuning pretrained model #489

Open poojitha0892 opened 3 days ago

poojitha0892 commented 3 days ago

Describe the bug I'm trying to train RT DETR model on custom dataset This is the error I'm getting Traceback (most recent call last): File "/RT-DETR/rtdetrv2_pytorch/tools/train.py", line 69, in main(args) File "/RT-DETR/rtdetrv2_pytorch/tools/train.py", line 39, in main solver.fit() File "/RT-DETR/rtdetrv2_pytorch/tools/../src/solver/det_solver.py", line 20, in fit self.train() File "/RT-DETR/rtdetrv2_pytorch/tools/../src/solver/_solver.py", line 69, in train self._setup() File "/RT-DETR/rtdetrv2_pytorch/tools/../src/solver/_solver.py", line 41, in _setup self.load_tuning_state(self.cfg.tuning) File "/RT-DETR/rtdetrv2_pytorch/tools/../src/solver/_solver.py", line 163, in load_tuning_state stat, infos = self._matched_state(module.state_dict(), state['ema']['module']) AttributeError: 'NoneType' object has no attribute 'state_dict'

I tried to print cfg in the code cfg: {'task': 'detection', '_model': None, '_postprocessor': None, '_criterion': None, '_optimizer': None, '_lr_scheduler': None, '_lr_warmup_scheduler': None, '_train_dataloader': None, '_val_dataloader': None, '_ema': None, '_scaler': None, '_train_dataset': None, '_val_dataset': None, '_collate_fn': None, '_evaluator': None, '_writer': None, 'num_workers': 0, 'batch_size': None, '_train_batch_size': None, '_val_batch_size': None, '_train_shuffle': None, '_val_shuffle': None, 'resume': None, 'tuning': 'https://github.com/lyuwenyu/storage/releases/download/v0.1/rtdetrv2_r101vd_6x_coco_from_paddle.pth', 'epoches': None, 'last_epoch': -1, 'use_amp': True, 'use_ema': False, 'ema_decay': 0.9999, 'ema_warmups': 2000, 'sync_bn': False, 'clip_max_norm': 0.0, 'find_unused_parameters': None, 'seed': 0, 'print_freq': None, 'checkpoint_freq': 1, 'output_dir': 'RT-DETR', 'summary_dir': None, 'device': '', 'yaml_cfg': {'task': 'detection', 'evaluator': {'type': 'CocoEvaluator', 'iou_types': ['bbox']}, 'num_classes': 9, 'remap_mscoco_category': True, 'train_dataloader': {'type': 'DataLoader', 'dataset': {'type': 'CocoDetection', 'img_folder': 'images/train/', 'ann_file': 'annotations/train.json', 'return_masks': False, 'transforms': {'type': 'Compose', 'ops': None}}, 'shuffle': True, 'num_workers': 2, 'drop_last': True, 'collate_fn': {'type': 'BatchImageCollateFuncion'}}, 'val_dataloader': {'type': 'DataLoader', 'dataset': {'type': 'CocoDetection', 'img_folder': 'images/val/', 'ann_file': 'annotations/val.json', 'return_masks': False, 'transforms': {'type': 'Compose', 'ops': None}}, 'shuffle': False, 'num_workers': 2, 'drop_last': False, 'collate_fn': {'type': 'BatchImageCollateFuncion'}}, 'config': '/data/factstream/poojitha/RT-DETR/rtdetrv2_pytorch/configs/dataset/coco_detection.yml', 'tuning': 'https://github.com/lyuwenyu/storage/releases/download/v0.1/rtdetrv2_r101vd_6x_coco_from_paddle.pth', 'seed': 0, 'use_amp': True, 'output_dir': 'RT-DETR', 'test_only': False, 'print_method': 'builtin', 'print_rank': 0}} here _model is None

To Reproduce This is train cmd - CUDA_VISIBLE_DEVICES=2,3 torchrun --master_port=9909 --nproc_per_node=2 tools/train.py -c ./RT-DETR/rtdetrv2_pytorch/configs/dataset/coco_detection.yml -t https://github.com/lyuwenyu/storage/releases/download/v0.1/rtdetrv2_r101vd_6x_coco_from_paddle.pth --use-amp --seed=0 --output-dir 'RT-DETR'

I want to finetune RT-DETRv2-X on custom dataset, Please help resolve this issue

lyuwenyu commented 3 days ago

I did not find any errors using the default config. see below log info. 

(py310) λ instance-3bwob41y-12 /paddle/workspace/repos/RT-DETR/rtdetrv2_pytorch python tools/train.py -c configs/rtdetrv2/rtdetrv2_r101vd_6x_coco.yml -t   https://github.com
/lyuwenyu/storage/releases/download/v0.1/rtdetrv2_r101vd_6x_coco_from_paddle.pth 
Not init distributed mode.
cfg:  {'task': 'detection', '_model': None, '_postprocessor': None, '_criterion': None, '_optimizer': None, '_lr_scheduler': None, '_lr_warmup_scheduler': None, '_train_dataloader': None, '_val_dataloader': None, '_ema': None, '_scaler': None, '_train_dataset': None, '_val_dataset': None, '_collate_fn': None, '_evaluator': None, '_writer': None, 'num_workers': 0, 'batch_size': None, '_train_batch_size': None, '_val_batch_size': None, '_train_shuffle': None, '_val_shuffle': None, 'resume': None, 'tuning': 'https://github.com/lyuwenyu/storage/releases/download/v0.1/rtdetrv2_r101vd_6x_coco_from_paddle.pth', 'epoches': 72, 'last_epoch': -1, 'use_amp': False, 'use_ema': True, 'ema_decay': 0.9999, 'ema_warmups': 2000, 'sync_bn': True, 'clip_max_norm': 0.1, 'find_unused_parameters': False, 'seed': None, 'print_freq': 100, 'checkpoint_freq': 1, 'output_dir': './output/rtdetrv2_r101vd_6x_coco', 'summary_dir': None, 'device': '', 'yaml_cfg': {'task': 'detection', 'evaluator': {'type': 'CocoEvaluator', 'iou_types': ['bbox']}, 'num_classes': 80, 'remap_mscoco_category': True, 'train_dataloader': {'type': 'DataLoader', 'dataset': {'type': 'CocoDetection', 'img_folder': './dataset/coco/train2017/', 'ann_file': './dataset/coco/annotations/instances_train2017.json', 'return_masks': False, 'transforms': {'type': 'Compose', 'ops': [{'type': 'RandomPhotometricDistort', 'p': 0.5}, {'type': 'RandomZoomOut', 'fill': 0}, {'type': 'RandomIoUCrop', 'p': 0.8}, {'type': 'SanitizeBoundingBoxes', 'min_size': 1}, {'type': 'RandomHorizontalFlip'}, {'type': 'Resize', 'size': [640, 640]}, {'type': 'SanitizeBoundingBoxes', 'min_size': 1}, {'type': 'ConvertPILImage', 'dtype': 'float32', 'scale': True}, {'type': 'ConvertBoxes', 'fmt': 'cxcywh', 'normalize': True}], 'policy': {'name': 'stop_epoch', 'epoch': 71, 'ops': ['RandomPhotometricDistort', 'RandomZoomOut', 'RandomIoUCrop']}}}, 'shuffle': True, 'num_workers': 4, 'drop_last': True, 'collate_fn': {'type': 'BatchImageCollateFuncion', 'scales': [480, 512, 544, 576, 608, 640, 640, 640, 672, 704, 736, 768, 800], 'stop_epoch': 71}, 'total_batch_size': 16}, 'val_dataloader': {'type': 'DataLoader', 'dataset': {'type': 'CocoDetection', 'img_folder': './dataset/coco/val2017/', 'ann_file': './dataset/coco/annotations/instances_val2017.json', 'return_masks': False, 'transforms': {'type': 'Compose', 'ops': [{'type': 'Resize', 'size': [640, 640]}, {'type': 'ConvertPILImage', 'dtype': 'float32', 'scale': True}]}}, 'shuffle': False, 'num_workers': 4, 'drop_last': False, 'collate_fn': {'type': 'BatchImageCollateFuncion'}, 'total_batch_size': 32}, 'print_freq': 100, 'output_dir': './output/rtdetrv2_r101vd_6x_coco', 'checkpoint_freq': 1, 'sync_bn': True, 'find_unused_parameters': False, 'use_amp': False, 'scaler': {'type': 'GradScaler', 'enabled': True}, 'use_ema': True, 'ema': {'type': 'ModelEMA', 'decay': 0.9999, 'warmups': 2000}, 'epoches': 72, 'clip_max_norm': 0.1, 'optimizer': {'type': 'AdamW', 'params': [{'params': '^(?=.*backbone)(?!.*norm|bn).*$', 'lr': 1e-06}, {'params': '^(?=.*(?:encoder|decoder))(?=.*(?:norm|bn)).*$', 'weight_decay': 0.0}], 'lr': 0.0001, 'betas': [0.9, 0.999], 'weight_decay': 0.0001}, 'lr_scheduler': {'type': 'MultiStepLR', 'milestones': [1000], 'gamma': 0.1}, 'lr_warmup_scheduler': {'type': 'LinearWarmup', 'warmup_duration': 2000}, 'model': 'RTDETR', 'criterion': 'RTDETRCriterionv2', 'postprocessor': 'RTDETRPostProcessor', 'use_focal_loss': True, 'eval_spatial_size': [640, 640], 'RTDETR': {'backbone': 'PResNet', 'encoder': 'HybridEncoder', 'decoder': 'RTDETRTransformerv2'}, 'PResNet': {'depth': 101, 'variant': 'd', 'freeze_at': 0, 'return_idx': [1, 2, 3], 'num_stages': 4, 'freeze_norm': True, 'pretrained': True}, 'HybridEncoder': {'in_channels': [512, 1024, 2048], 'feat_strides': [8, 16, 32], 'hidden_dim': 384, 'use_encoder_idx': [2], 'num_encoder_layers': 1, 'nhead': 8, 'dim_feedforward': 2048, 'dropout': 0.0, 'enc_act': 'gelu', 'expansion': 1.0, 'depth_mult': 1, 'act': 'silu'}, 'RTDETRTransformerv2': {'feat_channels': [384, 384, 384], 'feat_strides': [8, 16, 32], 'hidden_dim': 256, 'num_levels': 3, 'num_layers': 6, 'num_queries': 300, 'num_denoising': 100, 'label_noise_ratio': 0.5, 'box_noise_scale': 1.0, 'eval_idx': -1, 'num_points': [4, 4, 4], 'cross_attn_method': 'default', 'query_select_method': 'default'}, 'RTDETRPostProcessor': {'num_top_queries': 300}, 'RTDETRCriterionv2': {'weight_dict': {'loss_vfl': 1, 'loss_bbox': 5, 'loss_giou': 2}, 'losses': ['vfl', 'boxes'], 'alpha': 0.75, 'gamma': 2.0, 'matcher': {'type': 'HungarianMatcher', 'weight_dict': {'cost_class': 2, 'cost_bbox': 5, 'cost_giou': 2}, 'alpha': 0.25, 'gamma': 2.0}}, '__include__': ['../dataset/coco_detection.yml', '../runtime.yml', './include/dataloader.yml', './include/optimizer.yml', './include/rtdetrv2_r50vd.yml'], 'config': 'configs/rtdetrv2/rtdetrv2_r101vd_6x_coco.yml', 'tuning': 'https://github.com/lyuwenyu/storage/releases/download/v0.1/rtdetrv2_r101vd_6x_coco_from_paddle.pth', 'test_only': False, 'print_method': 'builtin', 'print_rank': 0}}
Start training
Load PResNet101 state_dict
tuning checkpoint from https://github.com/lyuwenyu/storage/releases/download/v0.1/rtdetrv2_r101vd_6x_coco_from_paddle.pth
Load model.state_dict, {'missed': [], 'unmatched': []}
Initial lr: [1e-06, 0.0001, 0.0001]
building train_dataloader with batch_size=16...
loading annotations into memory...
poojitha0892 commented 2 days ago

image In this part of code in /RT-DETR/rtdetrv2_pytorch/src/solver/_solver.py in the config _model is None , module will also be None and when trying to load module.state_dict(), it'll throw error right? stat, infos = self._matched_state(module.state_dict(), state['ema']['module']) AttributeError: 'NoneType' object has no attribute 'state_dict'