search

esdolo / ImmortalTracker

114 stars 20 forks source link

Using 20Hz #7

Open friedang opened 1 week ago

friedang commented 1 week ago

Hi @ImmortalTracker,

Many thanks for your work! I tried to use your tracker for Nuscenes mode 20Hz but run into several issues. Was the implementation not finished for this or are there different steps to take than for 2Hz?

Did you intend to create results for 2Hz and 20Hz and then merge such? I think keyframes are exluded in your results for 20Hz. But I am not sure if merging works in terms of track id.

Errors I encountered:

  1. Missing 20hz settings in main_nuscenes and no Dataloader for 20Hz
  2. When using 10Hz or normal NuscDataloader I get: `TYPE car SEQ 0 Frame 371 / 428 TYPE car SEQ 0 Frame 381 / 428 IndexError('list index out of range')

    /workspace/immo/data_loader/nuscenes_loader.py(170)next() 169 result = dict() --> 170 result['time_stamp'] = self.time_stamps[self.cur_frame] * 1e-6 171 ego = self.ego_info[str(self.cur_frame)]

ipdb> self.cur_frame 388`

If I ignore skip such frames my in either Loader, my result.json does not include keyframes as keys in the dict.

Best wishes

esdolo commented 1 week ago

Hi @friedang ,

  1. For nuScenes, we found the use of non-key frames (key frames = 2Hz frames) did not bring benefits (for evaluation on key-frames). Therefore, we did not debug the code under the 20Hz setting in the later code versions. However, a few adjustments should make the code work under 20Hz.
  2. No, we do not merge tracking results of 2Hz/20Hz. A 20Hz inference will provide complete tracking results at 20Hz.
  3. About the posted errors, did you create nuScenes data / detection results with mode=20Hz? The frame rate of dataloader/tracking is decided by the loaded ts/pose/info/detection data. NuScenesLoader10Hz can work under 20Hz when loading 20Hz data. Loading 20Hz data in main_nuscenes and using NuScenesLoader10Hz should work.
  4. Finally, you can find an implementation of tracking under 20Hz in nuScenes in https://github.com/tusen-ai/SimpleTrack.