Error when submitted zipped json file in nuscenes Tracking challenge

[xjturjc]

After I submitted the zipped json file, an error was reported during the execution, and the error was displayed as follows:

metrics_summary = nusc_eval.main() File “python3.8/site-packages/nuscenes/eval/tracking/evaluate.py”, line 204, in main metrics, metric_data_list = self.evaluate() File " python3.8/site-packages/nuscenes/eval/tracking/evaluate.py", line 135, in evaluate accumulate_class(class_name) File " python3.8/site-packages/nuscenes/eval/tracking/evaluate.py", line 131, in accumulate_class curr_md = currev.accumulate() File " python3.8/site-packages/nuscenes/eval/tracking/algo.py", line 140, in accumulate acc, = self.accumulate_threshold(threshold) File " python3.8/site-packages/nuscenes/eval/tracking/algo.py", line 292, in accumulate_threshold acc_merged = MOTAccumulatorCustom.merge_event_dataframes(accs) File " python3.8/site-packages/nuscenes/eval/tracking/mot.py", line 100, in merge_event_dataframes df = df.events File " python3.8/site-packages/nuscenes/eval/tracking/mot.py", line 61, in events self.cached_events_df = MOTAccumulatorCustom.new_event_dataframe_with_data(self._indices, self._events) File " python3.8/site-packages/nuscenes/eval/tracking/mot.py", line 37, in new_event_dataframe_with_data df = pd.DataFrame(events, index=idx, columns=[‘Type’, ‘OId’, ‘HId’, ‘D’]) File " python3.8/site-packages/pandas/core/frame.py", line 746, in init mgr = init_dict(data, index, columns, dtype=dtype) File " python3.8/site-packages/pandas/core/dtypes/cast.py", line 1175, in construct_1d_arraylike_from_scalar dtype=dtype.dtype AttributeError: type object ‘object’ has no attribute ‘dtype’

Searching on the Internet, it is said that it is a pandas version problem, and the 0.25 version will report an error. python3.8/site-packages/pandas/core/internals/construction.py Old version: nan_dtype = object New version: nan_dtype = np.dtype(“object”)

I use pandas 1.4.4 locally, and the eval is normal. After changing it to ‘nan_dtype = object’, the above error will indeed be reported.

It seems that upgrading the pandas version can solve this problem, or changing the above line of code, but the nuscenes test is submitted, and the environment is on the eval server, so we can’t modify it. I would like to ask if anyone who participated in the tracking challenges encountered this problem?

Thanks!

[whyekit-motional]

@xjturjc I believe the eval server uses pandas=1.x, so you should be fine if you perform the evaluation on the server

[xjturjc]

@xjturjc I believe the eval server uses pandas=1.x, so you should be fine if you perform the evaluation on the server

@whyekit-motional Thanks for your reply. But the error happened after I submitted it to the server. The above log is the Stderr file on the server. My team name is VPF. Can the administrator help me to take a look?

Thanks a lot!

[whyekit-motional]

@xjturjc I've restarted the server - do try again in a few hours, and let me know if you still face this issue

[xjturjc]

@xjturjc I've restarted the server - do try again in a few hours, and let me know if you still face this issue

Oh, I submited my method and default Bevfusion test result files again, It still report the error before

[whyekit-motional]

@xjturjc the server seems to be working now:

Can you give it another shot?

[xjturjc]

@xjturjc the server seems to be working now:

Can you give it another shot?

Thanks for your reply, I don’t know why the submission was successful this time. I submitted my method and BevFusion test result files again just now, but it still failed. The error reported was still a pandas and numpy version problem, the same as before.

Oh, is it possible that if you submit many times in a row, there will be one success?

Thanks a lot!

[whyekit-motional]

That is weird :thinking: I've reached out to the EvalAI team to check on this

I'll let you know as soon as they reply

[xjturjc]

Thanks, looking forward to your good news.

[xjturjc]

That is weird 🤔 I've reached out to the EvalAI team to check on this

I'll let you know as soon as they reply

The EvalAI server is OK now, Thanks for your replies for these days. Thanks sincerely!