search

tusen-ai / SimpleTrack

MIT License
333 stars 63 forks source link

Question about config on nuscenes #24

Closed liuchenxv closed 1 year ago

liuchenxv commented 2 years ago

Hello, It seems that the data association is "one-stage" just using the high confidence threshold according to the configure file on nuscenes(the redundancy mod is default). Does it mean "one-stage" data association have best performance on nuscenes, rather than "two-stage" association which is used on waymo dataset? So Could you explain why, Is it related to frequency of Datasets? Thank you

ziqipang commented 2 years ago

Hi @liuchenxv , thanks for the question! To briefly answer your question, yes, we do not have the two-stage association in the config file. To address your full concern, the following information and clarifications might be helpful.

  1. The difference in using two-stage association or not is driven by the different metrics on Waymo and NuScenes. Waymo uses MOTA as the main metric, requiring a high score threshold to filter out the low quality detections, thus need a two-stage association with low score threshold to save the early termination. However, NuScenes takes AMOTA as the main metric, which requires the outputting of both low&high quality boxes, thus we uses a super low score threshold and output almost all the detections. That's why we don't use two-stage association in NuScenes, because the score threshold cannot get even lower.

  2. For the 10Hz experiments in NuScenes, you can consider that we are also using the two-stage association in a way. We use the detections on non-key frames to extend the tracklets. You may refer to this function for details. In nuscenes inference, you can find the way to run SimpleTrack on NuScenes with 10Hz data and track the function step by step.

liuchenxv commented 2 years ago

Thanks.I get it.