KeyError: 'detections' - error with loading detections from dataset

[HelenTsvetkova]

Hello! I am trying to run demo/demo_mot_vis.py with my own bounding boxes from MTA dataset. Authors of MTA dataset provide script to convert it into COCO format. I run this script and it created dir with images and file coords.json with information about bboxes. I convert it from .json to .pkl (coords.pkl) format just as:

import pickle
# Read as bytes test
with open('test/coords.json', 'rb') as f:
    data =  f.read()
# Save as pickle
with open('test/coords.pkl', 'wb') as f:
    pickle.dump(data, f)

Then i edited file configs/mot/deepsort/deepsort_faster-rcnn_fpn_4e_mot17-public-half.py by changing data_root and detection_file to my custom (path to coords.pkl).

Then i run demo and received error:

'detections'
  File "/home/helen/tracking_projects/mmtracking-master/mmtrack/datasets/pipelines/loading.py", line 101, in __call__
    outs_det = results2outs(bbox_results=results['detections'])
  File "/home/helen/tracking_projects/mmtracking-master/mmtrack/apis/inference.py", line 108, in inference_mot
    data = test_pipeline(data)
  File "/home/helen/tracking_projects/mmtracking-master/demo/demo_mot_vis.py", line 81, in main
    result = inference_mot(model, img, frame_id=i)
  File "/home/helen/tracking_projects/mmtracking-master/demo/demo_mot_vis.py", line 106, in <module>
    main()

File coords.json have the following structure:

What i've done wrong..? Maybe I tottaly misunderstand something.

[dyhBUPT]

Hi, the reason may be that the "detections" is not included in the inference.py. (variable data in function inference_mot) For demo_mot_vis.py, you are suggested to use the "private config", not the public one.

In your case, maybe you want to use your own detections. You can use the tools/test.py to run inference.

[HelenTsvetkova]

Yes, I want to use my own detections as i mentioned above. I tryed to use "public config" like sugested here issue#127. I want to try tracking algorithms avoiding the detector stage of operation. Just tracking, using pre-detected bboxes. Do i need to run tools/test.py to achieve my goal..?

[dyhBUPT]

Yes, if you want to use the "public config", please use tools/test.py instead of demo_mot_vis.py. Please let me know if this helps.

Best wishes.

[HelenTsvetkova]

Thank you for your response! I created new config file :

_base_ = ['../mot/deepsort/deepsort_faster-rcnn_fpn_4e_mot17-private-half.py']

dataset_type = 'CocoDataset'
classes = ('person', 'background')
root_path = "/home/helen/DataSets/MTA/coco_MTA_ext_short/"
data = dict(
    samples_per_gpu=2,
    workers_per_gpu=2,
    train=dict(
        type=dataset_type,
        classes=classes,
        ann_file=root_path+'train/coords.pkl',
        img_prefix=root_path+'train/images/'),
    val=dict(
        type=dataset_type,
        classes=classes,
        ann_file=root_path+'test/coords.pkl',
        img_prefix=root_path+'test/images/',),
    test=dict(
        type=dataset_type,
        classes=classes,
        ann_file=root_path+'test/coords.pkl',
        img_prefix=root_path+'test/images/',)
        )

Then I run tools/test.py with new config file and got error:

[dyhBUPT]

Hi, It may be caused by that you are running a mot tracker DeepSORT but set dataset_type = CocoDataset. Use the 'MOTChallengeDataset' instead would help.

[HelenTsvetkova]

May be.. I changed dataset type to MOTChallengeDataset and got this:

[dyhBUPT]

Please show the complete error information.

One thing you should note is, you should use the public config instead of private config when you provide additional detection results.

[HelenTsvetkova]

My config file:

_base_ = ['../mot/deepsort/deepsort_faster-rcnn_fpn_4e_mot17-public-half.py']

dataset_type = 'MOTChallengeDataset'
classes = ('person', 'background')
root_path = "/home/helen/DataSets/MTA/coco_MTA_ext_short/"
data = dict(
    samples_per_gpu=2,
    workers_per_gpu=2,
    train=dict(
        type=dataset_type,
        classes=classes,
        ann_file=root_path+'test/coords.json',
        detection_file=root_path + 'test/coords.pkl',
        img_prefix=root_path+'test/images/'),
    val=dict(
        type=dataset_type,
        classes=classes,
        ann_file=root_path+'test/coords.json',
        detection_file=root_path + 'test/coords.pkl',
        img_prefix=root_path+'test/images/',),
    test=dict(
        type=dataset_type,
        classes=classes,
        ann_file=root_path+'test/coords.json',
        detection_file=root_path + 'test/coords.pkl',
        img_prefix=root_path+'test/images/',)
        )

Errors:

Traceback (most recent call last):
  File "/home/helen/.vscode/extensions/ms-python.python-2022.6.3/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_frame.py", line 948, in trace_dispatch
    self.do_wait_suspend(thread, frame, event, arg)
  File "/home/helen/.vscode/extensions/ms-python.python-2022.6.3/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_frame.py", line 164, in do_wait_suspend
    self._args[0].do_wait_suspend(*args, **kwargs)
  File "/home/helen/.vscode/extensions/ms-python.python-2022.6.3/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 2023, in do_wait_suspend
    keep_suspended = self._do_wait_suspend(thread, frame, event, arg, suspend_type, from_this_thread, frames_tracker)
  File "/home/helen/.vscode/extensions/ms-python.python-2022.6.3/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 2046, in _do_wait_suspend
    self._activate_gui_if_needed()
  File "/home/helen/.vscode/extensions/ms-python.python-2022.6.3/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 1571, in _activate_gui_if_needed
    activate_function()
  File "/home/helen/.vscode/extensions/ms-python.python-2022.6.3/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 1555, in <lambda>
    self.mpl_modules_for_patching = {"matplotlib": lambda: activate_matplotlib(do_enable_gui),
  File "/home/helen/.vscode/extensions/ms-python.python-2022.6.3/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydev_ipython/matplotlibtools.py", line 93, in activate_matplotlib
    gui, backend = find_gui_and_backend()
  File "/home/helen/.vscode/extensions/ms-python.python-2022.6.3/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydev_ipython/matplotlibtools.py", line 45, in find_gui_and_backend
    backend = matplotlib.rcParams['backend']
  File "/home/helen/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/matplotlib/_api/__init__.py", line 223, in __getattr__
    f"module {cls.__module__!r} has no attribute {name!r}")
AttributeError: module 'matplotlib' has no attribute 'rcParams'
/home/helen/tracking_projects/mmtracking-master/mmtrack/core/utils/misc.py:25: UserWarning: Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
  f'Setting OMP_NUM_THREADS environment variable for each process '
/home/helen/tracking_projects/mmtracking-master/mmtrack/core/utils/misc.py:35: UserWarning: Setting MKL_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
  f'Setting MKL_NUM_THREADS environment variable for each process '
loading annotations into memory...
Done (t=0.17s)
creating index...
index created!
Traceback (most recent call last):
  File "/home/helen/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/utils/registry.py", line 66, in build_from_cfg
    return obj_cls(**args)
  File "/home/helen/tracking_projects/mmtracking-master/mmtrack/datasets/mot_challenge_dataset.py", line 50, in __init__
    self.detections = self.load_detections(detection_file)
  File "/home/helen/tracking_projects/mmtracking-master/mmtrack/datasets/mot_challenge_dataset.py", line 70, in load_detections
    raise TypeError('detections must be a dict or a list.')
TypeError: detections must be a dict or a list.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/helen/anaconda3/envs/open-mmlab/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/helen/anaconda3/envs/open-mmlab/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/helen/.vscode/extensions/ms-python.python-2022.6.3/pythonFiles/lib/python/debugpy/__main__.py", line 45, in <module>
    cli.main()
  File "/home/helen/.vscode/extensions/ms-python.python-2022.6.3/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 444, in main
    run()
  File "/home/helen/.vscode/extensions/ms-python.python-2022.6.3/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 285, in run_file
    runpy.run_path(target_as_str, run_name=compat.force_str("__main__"))
  File "/home/helen/anaconda3/envs/open-mmlab/lib/python3.7/runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "/home/helen/anaconda3/envs/open-mmlab/lib/python3.7/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/home/helen/anaconda3/envs/open-mmlab/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "tools/test.py", line 14, in <module>
    from mmdet.apis import set_random_seed
  File "tools/test.py", line 151, in main
    dataset = build_dataset(cfg.data.test)
  File "/home/helen/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/mmdet/datasets/builder.py", line 82, in build_dataset
    dataset = build_from_cfg(cfg, DATASETS, default_args)
  File "/home/helen/anaconda3/envs/open-mmlab/lib/python3.7/site-packages/mmcv/utils/registry.py", line 69, in build_from_cfg
    raise type(e)(f'{obj_cls.__name__}: {e}')
TypeError: MOTChallengeDataset: detections must be a dict or a list.

To be honest, I don't have any detections_file, I have got only annotation file in 'json and .pkl format... Should I generate detection file from annotation file by myself?

[dyhBUPT]

Hi, Please make sure a correct format your detection_file. Codes of mot_challenge_dataset.py and mot2coco.py would help as a reference.