inference problems on nuscenes test set

[James-S-choi]

你好junjie，我是三维方向的萌新，问的问题可能比较简单，望不吝赐教，谢谢！！

我在推理阶段遇到了如下问题：

1. 将配置文件中的data-test字典中的ann_file修改为"nuscenes infos test.pkl"后，

报如下错误：

[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 6008/6008, 23.8 task/s, elapsed: 252s, ETA:     0s
Formating bboxes of pts_bbox
Start to convert detection format...
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 6008/6008, 20.0 task/s, elapsed: 301s, ETA:     0s
Results writes to /tmp/tmpcn3pefa4/results/pts_bbox/results_nusc.json
Evaluating bboxes of pts_bbox
Traceback (most recent call last):
  File "tools/test.py", line 289, in <module>
    main()
  File "tools/test.py", line 285, in main
    print(dataset.evaluate(outputs, **eval_kwargs))
  File "/opt/conda/lib/python3.7/site-packages/mmdet3d/datasets/nuscenes_dataset.py", line 509, in evaluate
    ret_dict = self._evaluate_single(result_files[name])
  File "/opt/conda/lib/python3.7/site-packages/mmdet3d/datasets/nuscenes_dataset.py", line 403, in _evaluate_single
    eval_set=eval_set_map[self.version],
KeyError: 'v1.0-test'
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 130354) of binary: /opt/conda/bin/python
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/launch.py", line 193, in <module>
    main()
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/launch.py", line 189, in main
    launch(args)
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/launch.py", line 174, in launch
    run(args)
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/run.py", line 713, in run
    )(*cmd_args)
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 131, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/opt/conda/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 261, in launch_agent
    failures=result.failures,
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
tools/test.py FAILED

请问这是什么原因呢？

2. 我看到配置文件中的验证集配置和测试集配置中的ann_file都是"nuscenes_infos_val.pkl"，这是为什么呢？不应该验证集和测试集一个是"nuscenes_infos_val.pkl"另一个是"nuscenes_infos_test.pkl"吗？

[junjie18]

@James-S-choi val.pkl本身是带gt的，所以可以本地评测。test是不带gt的，得去eval.ai上评测。

[James-S-choi]

@James-S-choi val.pkl本身是带gt的，所以可以本地评测。test是不带gt的，得去eval.ai上评测。

谢谢！