question about running pgd val and test code!

[MinHsuan123]

I finished training and want to val and test my model, but when it's finished, it doesn't show any mAP info!! only json, but I open json file,it look like this how can I know how much is mAP for my model? Thank you

[longyunf]

The evaluation code saves performance as metrics_summary.json in the folder out_dir. The code also prints a summary of performance. See https://github.com/longyunf/radiant/blob/cf5355396d42ef17940e29ef8f9e3cabfd8035c3/scripts/train_radiant_pgd.py#L397C1-L400

[stanny880913]

This is the results when I ran CUDA_VISIBLE_DEVICES=0 python scripts/train_radiant_pgd.py --do_eval --eval_set test，it did not show the summary of performance，why this happened?!

and the folder only had one json, no metrics_summary.json!! how can I fixed it~ thank you

[longyunf]

The evaluation is not conducted locally because there is no ground truth publicly available for test set. It is supposed to be conducted on nuScenes evaluation server with the generated result file.

[stanny880913]

The evaluation is not conducted locally because there is no ground truth publicly available for test set. It is supposed to be conducted on nuScenes evaluation server with the generated result file.

I can understand，and I found Nuscenes eval_ai，is this website you use to caculate mAP，Is the nuScenes detection challenge a challenge you participated in before? so I need to update results_nusc.json on the challenge，right ? but the results look like all Null，or this situation is normal? it's look like no bbox are detected !! thank you

[longyunf]

Yes. You need to zip the result file generated by evaluation code and submit it to the challenge.

[MinHsuan123]

Yes. You need to zip the result file generated by evaluation code and submit it to the challenge.

Sorry, but I ran the --eval test and always show the results like above!!!

The evaluation is not conducted locally because there is no ground truth publicly available for test set. It is supposed to be conducted on nuScenes evaluation server with the generated result file.

I can understand，and I found Nuscenes eval_ai，is this website you use to caculate mAP，Is the nuScenes detection challenge a challenge you participated in before? so I need to update results_nusc.json on the challenge，right ? but the results look like all Null，or this situation is normal? it's look like no bbox are detected !! thank you

How can I fix it? is there any parameters need to set up? or something went wrong? Thank you