search

jzhzhang / ROSEFusion

[SIGGRAPH 2021] ROSEFusion is proposed to tackle the difficulties in fast-motion camera tracking using random optimization with depth information only.
GNU General Public License v3.0
303 stars 48 forks source link

Masking human for indoor camera localization #39

Open bowieshi opened 3 weeks ago

bowieshi commented 3 weeks ago

Hi. If I have a RGBD video where a moving human is present. Is it possible for me to mask out the human by some segmentation model and then use ROSEFusion to estimate camera moving trajectory?

jzhzhang commented 3 weeks ago

ROSEFusion is a frame-to-model pipeline that requires depth information to compute the relative pose for the reconstruction model.

Thus, it depends on the remaining depth pixels after masking out the human. If a substantial number of depth pixels remain, ROSEFusion should function effectively.

bowieshi commented 3 weeks ago

@jzhzhang Thank you for your quick reply! Could you give me some hints where to modify your code in order to add a human mask? Thank you again!

jzhzhang commented 3 weeks ago

We do not have such an API to mask out pixels. ROSEFusion will read the depth map, and directly perform tracking. The related code can be found in here. You can directly add masked depth map, then run ROSEFusion.