search

TRI-ML / PF-Track

Implementation of PF-Track
Other
195 stars 25 forks source link

How can I apply this project from 3D to 2D? #37

Open PalmerXiao opened 1 month ago

PalmerXiao commented 1 month ago

Hello！I am very respected for your work.Now I am trying to achieve it to a multi-view 2D scene.Unfortunately it occurs some questions. ① in projects/configs/tracking/petr/f5_q500_800x320.py,which part I need to change?for instance Cam3DTracker->Cam2DTracker, PETRTransformerDecoder->DETRTransformerDecoder ,and backbone from vovnet->resnet,could I keep CPFPN as neck ,and SinePositionalEncoding3D->SinePositionalEncoding2D,and HungerianAssigner3D->HungerianAssigner2D. ②I need change your 3D-style detect head (PETR&DETR3D) to a 2D-style head such as DETR or Deformable DETR,what should I do? ③If I need to train this algorithm in my own 2D multi-view dataset such as MDMT(TMM 2023)https://github.com/VisDrone/Multi-Drone-Multi-Object-Detection-and-Tracking/tree/main ,what should I do? Thank you！

ziqipang commented 1 month ago

@StevenLingyuan Yes, I think all the things you are saying make sense. Please go ahead and try.

The only part I am not sure about is the motion prediction component in my part, which is easy to represent in 3D using xyz but tricky to represent in 2D due to perspective changes. So you could go ahead without my motion prediction component and adding it back later after the other parts are ready.