[Eval Track Results] ValueError: Length of names must match number of levels in MultiIndex.

[Jacky-gsq]

I run PYTHONPATH=. python tools/test.py [congfig] [ckpts] --eval bbox and get the bellow erros when I eval the tracking results:

`Accumulating metric data... Computing metrics for class bicycle...

Traceback (most recent call last):
File "tools/test.py", line 250, in main() File "tools/test.py", line 245, in main score_dict = dataset.evaluate(outputs, *eval_kwargs) File "/plugin/vip3d/dataset.py", line 1106, in evaluate results_dict = self._evaluate_single(result_files) File "plugin/vip3d/dataset.py", line 1028, in _evaluate_single metrics = nusc_eval.main() File "miniconda3/envs/mmdet3d/lib/python3.7/site-packages/nuscenes/eval/tracking/evaluate.py", line 204, in main metrics, metric_data_list = self.evaluate() File "miniconda3/envs/mmdet3d/lib/python3.7/site-packages/nuscenes/eval/tracking/evaluate.py", line 135, in evaluate accumulate_class(class_name) File "miniconda3/envs/mmdet3d/lib/python3.7/site-packages/nuscenes/eval/tracking/evaluate.py", line 131, in accumulate_class curr_md = curr_ev.accumulate() File "envs/mmdet3d/lib/python3.7/site-packages/nuscenes/eval/tracking/algo.py", line 123, in accumulate thresholds, recalls = self.compute_thresholds(gt_box_count) File "miniconda3/envs/mmdet3d/lib/python3.7/site-packages/nuscenes/eval/tracking/algo.py", line 308, in computethresholds , scores = self.accumulate_threshold(threshold=None) File "miniconda3/envs/mmdet3d/lib/python3.7/site-packages/nuscenes/eval/tracking/algo.py", line 278, in accumulate_threshold events = acc.events.loc[frame_id] File "miniconda3/envs/mmdet3d/lib/python3.7/site-packages/nuscenes/eval/tracking/mot.py", line 61, in events self.cached_events_df = MOTAccumulatorCustom.new_event_dataframe_with_data(self._indices, self._events) File "miniconda3/envs/mmdet3d/lib/python3.7/site-packages/nuscenes/eval/tracking/mot.py", line 36, in new_event_dataframe_with_data idx = pd.MultiIndex.from_tuples(indices, names=['FrameId', 'Event']) File "miniconda3/envs/mmdet3d/lib/python3.7/site-packages/pandas/core/indexes/multi.py", line 202, in new_meth return meth(self_or_cls, args, **kwargs) File "miniconda3/envs/mmdet3d/lib/python3.7/site-packages/pandas/core/indexes/multi.py", line 560, in from_tuples return cls.from_arrays(arrays, sortorder=sortorder, names=names) File "miniconda3/envs/mmdet3d/lib/python3.7/site-packages/pandas/core/indexes/multi.py", line 492, in from_arrays verify_integrity=False, File "miniconda3/envs/mmdet3d/lib/python3.7/site-packages/pandas/core/indexes/multi.py", line 331, in new result._set_names(names) File "miniconda3/envs/mmdet3d/lib/python3.7/site-packages/pandas/core/indexes/multi.py", line 1431, in _set_names "Length of names must match number of levels in MultiIndex." ValueError: Length of names must match number of levels in MultiIndex.`

[GentleSmile]

Hi, what is your package version of nuscenes-devkit and pandas? Will the same error occur when using the checkpoint we uploaded?

[Jacky-gsq]

The version of nuscenes-deckit is 1.1.2 and pandas is 1.3.5. I used the detr3d_resnet50.pth provided by you and then trained the vip3d 24 epochs, getting the above error when evaluating.

[GentleSmile]

We also encountered the same problem. It can be fixed by installing motmetrics<=1.1.3

[Jacky-gsq]

thanks, that solve the problem!