Open mmeendez8 opened 1 year ago
mmeendez8
commented
1 year ago I just debugged model output and I get the following:
print(result.video_data_samples[0].pred_instances.bboxes)
tensor([[-1.9546e+09, -5.3129e+09, -1.9546e+09, -5.3129e+09],
[ -inf, -7.4956e+09, inf, -7.4956e+09],
[ -inf, 1.6434e+09, inf, 1.6434e+09],
[ 4.2734e+09, 1.9240e+09, 4.2734e+09, 1.9240e+09]], device='cuda:0')Some more comments, I just got confused and thought the weight path of the detection model was burned in the config but I need to manually attach it. Everything works if you just use the proper command:
python demo/mot_demo.py \
demo/demo_mot.mp4 \
configs/bytetrack/bytetrack_yolox_x_8xb4-80e_crowdhuman-mot17halftrain_test-mot17halfval.py \
--checkpoint https://download.openmmlab.com/mmtracking/mot/bytetrack/bytetrack_yolox_x/bytetrack_yolox_x_crowdhuman_mot17-private-half_20211218_205500-1985c9f0.pth
--out output/bytetrack.mp4 \But problems arise with SORT style trackers where you have --detector and --reid flags. I get NaN errors again... it seems the model is being loaded in a different way but I cannot really see what is going on.
This command works without any trouble:
python demo/mot_demo.py \
demo/demo.mp4 \
configs/sort/sort_yolox_x_8xb4-80e_crowdhuman-mot17halftrain_test-mot17halfval.py \
--checkpoint https://download.openmmlab.com/mmtracking/mot/bytetrack/bytetrack_yolox_x/bytetrack_yolox_x_crowdhuman_mot17-private-half_20211218_205500-1985c9f0.pth \
--out output/sort_yolox.mp4But I try to do the same using the --detector flag I got NaN errors again because the model output is NaN:
python demo/mot_demo.py \
demo/demo.mp4 \
configs/sort/sort_yolox_x_8xb4-80e_crowdhuman-mot17halftrain_test-mot17halfval.py \
--detector https://download.openmmlab.com/mmtracking/mot/bytetrack/bytetrack_yolox_x/bytetrack_yolox_x_crowdhuman_mot17-private-half_20211218_205500-1985c9f0.pth \
--out output/sort_yolox.mp4This is the config file I am using for SORT, just modified it to add YOLOX as the detector:
_base_ = ['../yolox/yolox_x_8xb8-300e_coco.py'] # same as mmdet one
dataset_type = 'MOTChallengeDataset'
data_root = 'data/MOT17/'
img_scale = (1440, 800) # weight, height
batch_size = 4
detector = _base_.model
detector.pop('data_preprocessor')
detector.bbox_head.update(dict(num_classes=1))
detector.test_cfg.nms.update(dict(iou_threshold=0.7))
detector['init_cfg'] = dict(
type='Pretrained',
checkpoint= # noqa: E251
'https://download.openmmlab.com/mmdetection/v2.0/yolox/yolox_x_8x8_300e_coco/yolox_x_8x8_300e_coco_20211126_140254-1ef88d67.pth' # noqa: E501
)
del _base_.model
model = dict(
type='DeepSORT',
data_preprocessor=dict(
type='TrackDataPreprocessor',
pad_size_divisor=32,),
detector=detector,
tracker=dict(
type='SORTTracker',
motion=dict(type='KalmanFilter', center_only=False),
obj_score_thr=0.5,
match_iou_thr=0.5,
reid=None))
test_pipeline = [
dict(
type='TransformBroadcaster',
transforms=[
dict(type='LoadImageFromFile', backend_args=_base_.backend_args),
dict(type='Resize', scale=img_scale, keep_ratio=True),
dict(
type='Pad',
size_divisor=32,
pad_val=dict(img=(114.0, 114.0, 114.0))),
dict(type='LoadTrackAnnotations'),
]),
dict(type='PackTrackInputs')
]
val_dataloader = dict(
_delete_=True,
batch_size=1,
num_workers=2,
persistent_workers=True,
pin_memory=True,
drop_last=False,
# video_based
# sampler=dict(type='DefaultSampler', shuffle=False, round_up=False),
sampler=dict(type='TrackImgSampler'), # image_based
dataset=dict(
type=dataset_type,
data_root=data_root,
ann_file='annotations/half-val_cocoformat.json',
data_prefix=dict(img_path='train'),
test_mode=True,
pipeline=test_pipeline))
test_dataloader = val_dataloader
vis_backends = [dict(type='LocalVisBackend')]
visualizer = dict(
type='TrackLocalVisualizer', vis_backends=vis_backends, name='visualizer')
del detector
Describe the bug
I am trying to rum tracking with bytetrack model that uses YOLOX but I am not able to get it to work. Reproduction
Environment
Error traceback