search

SysCV / qd-3dt

Official implementation of Monocular Quasi-Dense 3D Object Tracking, TPAMI 2022
https://eborboihuc.github.io/QD-3DT/
BSD 3-Clause "New" or "Revised" License
512 stars 99 forks source link

the 3d box of the car is not`t very good #10

Closed tust13018211 closed 3 years ago

tust13018211 commented 3 years ago

Thanks for your work, I use my data to test the code. But I find the 3d box is non`t very good, I want to konw the reason? GIF

tust13018211 commented 3 years ago

the model I used is kitti，not trained by myself，maybe this cause the bad effect

eborboihuc commented 3 years ago

Thanks for trying our method and applying our code to your data.

You may want to take a look at the input data, especially on the camera matrix. From the visualization, the camera you used might not share the same camera intrinsic with KITTI. Also, it seems the camera extrinsic is not updating or not reflecting your ego-motion.

tust13018211 commented 3 years ago

I really appreciate your rapid reply! Firtstly, the camera intrinsic and camera extrinsic has been adapted to our realstic sence; Secondly, would you please tell me that if the camera extrinsic is updated in real time? As you can see, the trace of the car seems correct, so i think the camera extrinsic is right, but the 3d box is notn`t very good. Thank you again and expected for your reply.

GIF GIF2

tust13018211 commented 3 years ago

I have a another quesion about the camera extrinsic, why in the kitti data, the global position didn`t multiply camera extrinsic, only the roattion multiply camera extrinsic. but in waymo and nuSceness data, all the global pose and rotation multiply camera extrinsic？ I have a little confuse about this

tobiasfshr commented 3 years ago

Hi,

thanks for trying out our work! In the top video, the cars are moving at a very similar speed like the ego car, therefore the visualization doesn't really say much about the extrinsics (note that the BEV visualization shows the scene in the current camera frame, not in world frame!). A more reliable way to check the extrinsics would be to take a sequence where the ego car passes a parked car, and see if the position of the parked car stays constant in world frame. The camera position will be updated at each input frame if that's your question regarding real-time update. Your second question seems confusing, the camera extrinsic is the global position and rotation. It relates your current camera coordinate frame to some global ('world') frame. Finally, i'd try to use the nuScenes model, since this one was trained with much more real-world data and thus may generalize better to your scenario. The best would probably still be to fine-tune it on some labelled data from your camera setup.

Best, Tobias