SysCV / shift-dev

SHIFT Dataset DevKit - CVPR2022
https://www.vis.xyz/shift
MIT License
103 stars 10 forks source link

[WIP] Add SHIFT reference evaluators #45

Open suniique opened 1 year ago

suniique commented 1 year ago
Task Metrics Status
Semantic segmentation mIoU, mAcc, fwIoU, fwAcc [x]
Object detection 2D COCO (mAP, AP50 AP75) [x]
Instance segmentation COCO (mAP, AP50 AP75) [x]
Depth estimation MAE, Delta_{1,2,5}, SILog [x]
Optical flow EPE [x]
MOT MOTA, HOTA, IDSw [ ]
Object detection 3D nuScene (mAP, AP50 AP75) [ ]
MOT 3D nuScene (MOTA, HOTA, IDSw) [ ]
Trajectory forecasting EPE, APE [ ]
Pose estimation (human) PCK [ ]
Dump to eval.ai format [x]