search

traveller59 / second.pytorch

SECOND for KITTI/NuScenes object detection
MIT License
1.72k stars 721 forks source link

Evaluation on Mini dataset gives ""Samples in split doesn't match samples in predictions."" #307

Open chowkamlee81 opened 4 years ago

chowkamlee81 commented 4 years ago

For training and val dataset , evaluation toolkit works but when i start evaluating on v1.0 mini dataset gives

""Samples in split doesn't match samples in predictions.""

Kindly help

tjucwb commented 4 years ago

Hi, dude, have you solved this problem, I also encountered this problem, THANKS

tjucwb commented 4 years ago

I solved that, just modify line 308 in nuscenes_dataset.py eval_set_map, from "v1.0-mini": "mini_train" to "v1.0-mini": "mini_val", additionally, the size of either set, train_scenes and val_scenes can not be zero

3846chs commented 1 year ago

See tools/dist_test.py#L201.

This problem occurs because v1.0-trainval dataset is loaded. (not v1.0-mini)

Change 

result_dict, _ = dataset.evaluation(copy.deepcopy(predictions), output_dir=args.work_dir, testset=args.testset)

to

dataset.version = "v1.0-mini"
result_dict, _ = dataset.evaluation(copy.deepcopy(predictions), output_dir=args.work_dir, testset=args.testset)

This command causes dataset.version to change from v1.0-trainval to v1.0-mini.

wongyibin commented 1 year ago

I used "v1.0-mini“ and meet this problem. I modify this line : eval_set_map = { 'v1.0-mini': 'mini_val', 'v1.0-trainval': 'val', } to : eval_set_map = { 'v1.0-mini': 'mini_train', 'v1.0-trainval': 'val', }

Byte247 commented 7 months ago

hmm still happens to me even though it already says: "v1.0-mini": "mini_val"