Question about Argo2 dataset.

liyiheng0913 commented 8 months ago

Hi, thanks for your excellent work! I want to compare your work with mine on argoverse2 dataset, but only find mAP results in the paper. Could you please give me the complete results on av2 ? Thanks a lot !

junjie18 commented 8 months ago

@liyiheng0913 hi, this is our result

ARTICULATED_BUS: {"AP":0.247,"ATE":0.901,"ASE":0.214,"AOE":0.145,"CDS":0.189}, 
BICYCLE: {"AP":0.403,"ATE":0.52,"ASE":0.28,"AOE":0.775,"CDS":0.298}, 
BICYCLIST: {"AP":0.502,"ATE":0.329,"ASE":0.288,"AOE":0.409,"CDS":0.404}, 
BOLLARD: {"AP":0.496,"ATE":0.377,"ASE":0.453,"AOE":0.842,"CDS":0.345}, 
BOX_TRUCK: {"AP":0.384,"ATE":0.643,"ASE":0.235,"AOE":0.121,"CDS":0.308}, 
BUS: {"AP":0.415,"ATE":0.677,"ASE":0.189,"AOE":0.13,"CDS":0.336}, 
CONSTRUCTION_BARREL: {"AP":0.622,"ATE":0.346,"ASE":0.273,"AOE":0.865,"CDS":0.473}, 
CONSTRUCTION_CONE: {"AP":0.595,"ATE":0.344,"ASE":0.425,"AOE":0.807,"CDS":0.425}, 
DOG: {"AP":0.215,"ATE":0.841,"ASE":0.498,"AOE":0.878,"CDS":0.129}, 
LARGE_VEHICLE: {"AP":0.068,"ATE":0.751,"ASE":0.314,"AOE":0.364,"CDS":0.05}, 
MESSAGE_BOARD_TRAILER: {"AP":0.0,"ATE":2.0,"ASE":1.0,"AOE":3.142,"CDS":0.0}, 
MOBILE_PEDESTRIAN_CROSSING_SIGN: {"AP":0.403,"ATE":0.215,"ASE":0.448,"AOE":0.627,"CDS":0.302}, 
MOTORCYCLE: {"AP":0.551,"ATE":0.447,"ASE":0.274,"AOE":0.492,"CDS":0.431}, 
MOTORCYCLIST: {"AP":0.582,"ATE":0.349,"ASE":0.285,"AOE":0.272,"CDS":0.476}, 
PEDESTRIAN: {"AP":0.612,"ATE":0.35,"ASE":0.25,"AOE":0.879,"CDS":0.468}, 
REGULAR_VEHICLE: {"AP":0.719,"ATE":0.407,"ASE":0.159,"AOE":0.127,"CDS":0.622}, 
SCHOOL_BUS: {"AP":0.473,"ATE":0.681,"ASE":0.184,"AOE":0.114,"CDS":0.384}, 
SIGN: {"AP":0.284,"ATE":0.416,"ASE":0.364,"AOE":0.629,"CDS":0.211}, 
STOP_SIGN: {"AP":0.547,"ATE":0.343,"ASE":0.396,"AOE":0.24,"CDS":0.43}, 
STROLLER: {"AP":0.046,"ATE":0.705,"ASE":0.471,"AOE":0.358,"CDS":0.032}, 
TRUCK: {"AP":0.214,"ATE":0.697,"ASE":0.203,"AOE":0.143,"CDS":0.171}, 
TRUCK_CAB: {"AP":0.236,"ATE":0.904,"ASE":0.341,"AOE":0.181,"CDS":0.169}, 
VEHICULAR_TRAILER: {"AP":0.238,"ATE":0.831,"ASE":0.279,"AOE":0.471,"CDS":0.171},
 WHEELCHAIR: {"AP":0.036,"ATE":0.269,"ASE":0.448,"AOE":1.463,"CDS":0.024}, 
WHEELED_DEVICE: {"AP":0.327,"ATE":0.496,"ASE":0.203,"AOE":0.816,"CDS":0.249}, 
WHEELED_RIDER: {"AP":0.162,"ATE":0.37,"ASE":0.363,"AOE":0.662,"CDS":0.121}, 
AVERAGE_METRICS: {"AP":0.361,"ATE":0.585,"ASE":0.34,"AOE":0.614,"CDS":0.278}

liyiheng0913 commented 7 months ago

Thanks for your reply! What's the image resolution and lidar voxel size when training CMT on Argoverse2 ?

junjie18 / CMT

Question about Argo2 dataset. #81