Changing detector of video models to yolox

[MiladGhorbaniG]

I tested SELSA with temporal ROI on my custom dataset but it doesn't work well on far objects, I tried to use YOLOX model instead of the Faster R-CNN detector, I changed the config to :

_base_ = [
    '../../_base_/models/yolox_x_8x8.py)',
    '../../_base_/datasets/my_dataset.py',
    '../../_base_/default_runtime.py'
]
model = dict(
    type='SELSA',
    detector=dict(
        roi_head=dict(
            type='SelsaRoIHead',
            bbox_roi_extractor=dict(
                type='TemporalRoIAlign',
                num_most_similar_points=2,
                num_temporal_attention_blocks=4,
                roi_layer=dict(
                    type='RoIAlign', output_size=7, sampling_ratio=2),
                out_channels=512,
                featmap_strides=[16]),
            bbox_head=dict(
                type='SelsaBBoxHead',
                num_shared_fcs=3,
                aggregator=dict(
                    type='SelsaAggregator',
                    in_channels=1024,
                    num_attention_blocks=16)))))

# dataset settings
data = dict(
    val=dict(
        ref_img_sampler=dict(
            _delete_=True,
            num_ref_imgs=14,
            rame_range=[-15, 0],
            method='test_with_adaptive_stride')),
    test=dict(
        ref_img_sampler=dict(
            _delete_=True,
            num_ref_imgs=14,
            frame_range=[-15, 0],
            method='test_with_adaptive_stride')))

# optimizer
optimizer = dict(type='SGD', lr=0.002, momentum=0.9, weight_decay=0.0001)
optimizer_config = dict(
    _delete_=True, grad_clip=dict(max_norm=35, norm_type=2))
# learning policy
lr_config = dict(
    policy='step',
    warmup='linear',
    warmup_iters=500,
    warmup_ratio=1.0 / 3,
    step=[2, 5])
# runtime settings
total_epochs = 200
evaluation = dict(metric=['bbox'], interval=1000)

I faced the following error when I start to train with it:

2022-05-05 14:05:59,904 - mmtrack - INFO - Set random seed to 1726823364, deterministic: False
Traceback (most recent call last):
  File "/home/nvidia/miniconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/utils/registry.py", line 52, in build_from_cfg
    return obj_cls(**args)
TypeError: __init__() got an unexpected keyword argument 'roi_head'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/nvidia/miniconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/utils/registry.py", line 52, in build_from_cfg
    return obj_cls(**args)
  File "/home/nvidia/Downloads/mmtracking/mmtrack/models/vid/selsa.py", line 37, in __init__
    self.detector = build_detector(detector)
  File "/home/nvidia/miniconda3/envs/open-mmlab/lib/python3.7/site-packages/mmdet/models/builder.py", line 59, in build_detector
    cfg, default_args=dict(train_cfg=train_cfg, test_cfg=test_cfg))
  File "/home/nvidia/miniconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/utils/registry.py", line 212, in build
    return self.build_func(*args, **kwargs, registry=self)
  File "/home/nvidia/miniconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/cnn/builder.py", line 27, in build_model_from_cfg
    return build_from_cfg(cfg, registry, default_args)
  File "/home/nvidia/miniconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/utils/registry.py", line 55, in build_from_cfg
    raise type(e)(f'{obj_cls.__name__}: {e}')
TypeError: YOLOX: __init__() got an unexpected keyword argument 'roi_head'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tools/train.py", line 210, in <module>
    main()
  File "tools/train.py", line 182, in main
    model = build_model(cfg.model)
  File "/home/nvidia/Downloads/mmtracking/mmtrack/models/builder.py", line 35, in build_model
    return MODELS.build(cfg)
  File "/home/nvidia/miniconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/utils/registry.py", line 212, in build
    return self.build_func(*args, **kwargs, registry=self)
  File "/home/nvidia/miniconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/cnn/builder.py", line 27, in build_model_from_cfg
    return build_from_cfg(cfg, registry, default_args)
  File "/home/nvidia/miniconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/utils/registry.py", line 55, in build_from_cfg
    raise type(e)(f'{obj_cls.__name__}: {e}')
TypeError: SELSA: YOLOX: __init__() got an unexpected keyword argument 'roi_head'

Any suggestions?

[GT9505]

SELSA and SELSA with temporal ROI align need proposals generated by two stage detector. If you want to use yolox in vid, I suggest you use DFF or FGFA method