search

henry123-boy / SpaTracker

[CVPR 2024 Highlight] Official PyTorch implementation of SpatialTracker: Tracking Any 2D Pixels in 3D Space
Other
717 stars 25 forks source link

question about the paper #10

Open CHNxindong opened 5 months ago

CHNxindong commented 5 months ago

A good work！ When I read the paper, as marked in yellow below, I didn't find a counterpart in the code. Can you give me some suggestions. Looking forward to your reply! image

wzq20030207 commented 5 months ago

This is also what I want to ask. Hope anyone can answer this.

henry123-boy commented 5 months ago

Hi, thank you for noticing our work. During our training, we used the ground truth intrinsic and depth to calculate the ARAP loss (equ.4 in our paper), which is not appeared in our inference codes. Hope this helps

CHNxindong commented 5 months ago

Hi, thank you for noticing our work. During our training, we used the ground truth intrinsic and depth to calculate the ARAP loss (equ.4 in our paper), which is not appeared in our inference codes. Hope this helps

Thank your reply! I wonder if this part of the code(gt depth maps and camera intrinsic to unproject pixels into 3D space) will be open source.

sriramsk1999 commented 4 months ago

The pinhole camera equation is used to unproject a depth image/intrinsic to 3D. You can use this open3d api for the same.

TranHuuNhatHuy commented 3 months ago

Hello, thank you for this interesting project.

I would like to ask, what is the computational cost per a paired image? How does it depend on the number of tracking points? I could not find this information in the paper itself.

Also, what do you think about its effectiveness in extreme cases, such as tracking crossing pedestrians at a densely crowded intersection?