NotImplementedError: invalid shape: torch.Size([8, 256])

[carpeug]

Hi, I'm just trying load your model with the pretrained weights provided on BDD100K. I have modified dataset_to_num_classes in motr_co.py in this way: dataset_to_num_classes = { 'coco': 91, 'coco_panoptic': 250, 'e2e_mot': 1, 'e2e_bdd': 11, 'e2e_tao': 2000, 'e2e_bddcc': 100, 'e2e_dance': 1, 'e2e_joint': 1, 'e2e_static_mot': 1, 'e2e_all': 91, 'bdd100k_mot': 8 }

I have modified configs/motrv2ch_uni5cost3ggoon.args in this way: --meta_arch motr_unincost --dataset_file bdd100k_mot --epoch 20 --with_box_refine --lr_drop 8 --lr 2e-4 --lr_backbone 2e-5 --pretrained /content/drive/MyDrive/CO-MOT-main/comot_bdd100k.pth --batch_size 1 --sample_mode random_interval --sample_interval 10 --sampler_lengths 5 --merger_dropout 0 --dropout 0 --random_drop 0.1 --fp_ratio 0.3 --query_interaction_layer GQIM --num_queries 60 --append_crowd --use_checkpoint --mot_path /content/drive/MyDrive/waymo_open_dataset/coco_waymo/validation --match_type gmatch --g_size 3 --output_dir /content/drive/MyDrive/waymo_open_dataset/coco_waymo/validation/CO-MOT

However, when i run: /content/drive/MyDrive/CO-MOT-main/tools/simple_inference.sh /content/drive/MyDrive/CO-MOT-main/configs/motrv2ch_uni5cost3ggoon.args --resume /content/drive/MyDrive/CO-MOT-main/comot_bdd100k.pth

Where simple_inference.sh is the following: `#!/usr/bin/env bash

------------------------------------------------------------------------

Copyright (c) 2022 megvii-research. All Rights Reserved.

------------------------------------------------------------------------

set -x set -o pipefail

args=$(cat configs/motrv2.args)

args=$(cat $1) python3 /content/drive/MyDrive/CO-MOT-main/submit_dance.py ${args} --exp_name tracker --resume /content/drive/MyDrive/CO-MOT-main/comot_bdd100k.pth

`

I obtain the following error: + python3 /content/drive/MyDrive/CO-MOT-main/submit_dance.py --meta_arch motr_unincost --dataset_file bdd100k_mot --epoch 20 --with_box_refine --lr_drop 8 --lr 2e-4 --lr_backbone 2e-5 --pretrained /content/drive/MyDrive/CO-MOT-main/comot_bdd100k.pth --batch_size 1 --sample_mode random_interval --sample_interval 10 --sampler_lengths 5 --merger_dropout 0 --dropout 0 --random_drop 0.1 --fp_ratio 0.3 --query_interaction_layer GQIM --num_queries 60 --append_crowd --use_checkpoint --mot_path /content/drive/MyDrive/waymo_open_dataset/coco_waymo/validation --match_type gmatch --g_size 3 --output_dir /content/drive/MyDrive/waymo_open_dataset/coco_waymo/validation/CO-MOT --exp_name tracker --resume /content/drive/MyDrive/CO-MOT-main/comot_bdd100k.pth Namespace(lr=0.0002, lr_backbone_names=['backbone.0'], lr_backbone=2e-05, lr_linear_proj_names=['reference_points', 'sampling_offsets'], lr_linear_proj_mult=0.1, batch_size=1, weight_decay=0.0001, epochs=20, lr_drop=8, save_period=50, lr_drop_epochs=None, clip_max_norm=0.1, meta_arch='motr_unincost', sgd=False, with_box_refine=True, two_stage=False, accurate_ratio=False, dn_labelbook_size=91, dec_pred_class_embed_share=True, dec_pred_bbox_embed_share=True, fix_refpoints_hw=-1, two_stage_class_embed_share=False, two_stage_bbox_embed_share=False, use_dn=True, dn_number=100, dn_box_noise_scale=0.4, dn_label_noise_ratio=0.5, num_select=300, nms_iou_threshold=-1, frozen_weights=None, backbone='resnet50', enable_fpn=False, dilation=False, position_embedding='sine', position_embedding_scale=6.283185307179586, num_feature_levels=4, pe_temperatureH=20, pe_temperatureW=20, return_interm_indices=[0, 1, 2, 3], backbone_freeze_keywords=None, trans_mode='DeformableTransformer', enc_layers=6, dec_layers=6, dim_feedforward=1024, hidden_dim=256, dropout=0.0, nheads=8, num_queries=60, dec_n_points=4, enc_n_points=4, decoder_cross_self=False, sigmoid_attn=False, crop=False, cj=False, extra_track_attn=False, loss_normalizer=False, max_size=1333, val_width=800, filter_ignore=False, append_crowd=True, decoder_layer_noise=False, dln_xy_noise=0.2, dln_hw_noise=0.2, use_detached_boxes_dec_out=False, unic_layers=0, pre_norm=False, query_dim=4, transformer_activation='relu', num_patterns=0, use_deformable_box_attn=False, box_attn_type='roi_align', add_channel_attention=False, add_pos_value=False, random_refpoints_xy=False, two_stage_type='standard', two_stage_pat_embed=0, two_stage_add_query_num=0, two_stage_learn_wh=False, two_stage_keep_all_tokens=False, dec_layer_number=None, decoder_sa_type='sa', decoder_module_seq=['sa', 'ca', 'ffn'], embed_init_tgt=True, no_interm_box_loss=False, interm_loss_coef=1.0, masks=False, aux_loss=True, match_type='gmatch', mix_match=False, set_cost_class=2, set_cost_bbox=5, set_cost_giou=2, match_unstable_error=True, mask_loss_coef=1, dice_loss_coef=1, cls_loss_coef=2, bbox_loss_coef=5, giou_loss_coef=2, focal_alpha=0.25, dataset_file='bdd100k_mot', gt_file_train=None, gt_file_val=None, coco_path='/data/workspace/detectron2/datasets/coco/', coco_panoptic_path=None, remove_difficult=False, output_dir='/content/drive/MyDrive/waymo_open_dataset/coco_waymo/validation/CO-MOT', device='cuda', seed=42, resume='/content/drive/MyDrive/CO-MOT-main/comot_bdd100k.pth', start_epoch=0, eval=False, vis=False, num_workers=2, pretrained='/content/drive/MyDrive/CO-MOT-main/comot_bdd100k.pth', cache_mode=False, mot_path='/content/drive/MyDrive/waymo_open_dataset/coco_waymo/validation', det_db='', input_video='figs/demo.mp4', data_txt_path_train='./datasets/data_path/detmot17.train', data_txt_path_val='./datasets/data_path/detmot17.train', img_path='data/valid/JPEGImages/', query_interaction_layer='GQIM', sample_mode='random_interval', sample_interval=10, random_drop=0.1, fp_ratio=0.3, merger_dropout=0.0, update_query_pos=False, sampler_steps=None, sampler_lengths=[5], exp_name='tracker', memory_bank_score_thresh=0.0, memory_bank_len=4, memory_bank_type=None, memory_bank_with_self_attn=False, use_checkpoint=True, query_denoise=0.0, g_size=3, score_threshold=0.5, update_score_threshold=0.5, miss_tolerance=20, not_valid=True) /usr/local/lib/python3.10/dist-packages/torchvision/__init__.py <module 'torchvision' from '/usr/local/lib/python3.10/dist-packages/torchvision/__init__.py'> /usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead. warnings.warn( /usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum orNonefor 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passingweights=None. warnings.warn(msg) Training with Self-Cross Attention. loaded /content/drive/MyDrive/CO-MOT-main/comot_bdd100k.pth Skip loading parameter class_embed.0.weight, required shapetorch.Size([8, 256]), loaded shapetorch.Size([100, 256]). If you see this, your model does not fully load the pre-trained weight. Please make sure you set the correct --num_classes for your own dataset. load class_embed: class_embed.0.weight shape=torch.Size([100, 256]) Traceback (most recent call last): File "/content/drive/MyDrive/CO-MOT-main/submit_dance.py", line 318, in <module> detr = load_model(detr, args.resume) File "/content/drive/MyDrive/CO-MOT-main/util/tool.py", line 50, in load_model raise NotImplementedError('invalid shape: {}'.format(model_state_dict[k].shape)) NotImplementedError: invalid shape: torch.Size([8, 256])

Can you please help me?

[carpeug]

Hi, I solved it by simply specifying "--dataset_file e2e_bddcc" inside configs/motrv2ch_uni5cost3ggoon.args.