search

dcharatan / flowmap

Code for "FlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient Descent" by Cameron Smith*, David Charatan*, Ayush Tewari, and Vincent Sitzmann
https://cameronosmith.github.io/flowmap/
MIT License
872 stars 84 forks source link

Dynamic Setting #35

Closed skrya closed 3 months ago

skrya commented 3 months ago

Dear Authors,

Thanks for the amazing work! I know that this work deals with static setting. I am curious to know if this work can be extended to dynamic setting with optical and foreground and background (static) segmentation available? Please share your thoughts.

Thanks!

dcharatan commented 3 months ago

If dynamic parts of the scene were perfectly segmented and FlowMap's losses were masked accordingly, FlowMap would work on dynamic videos (at least for pose estimation). This shouldn't be too hard to implement—you could probably load the static/dynamic mask in the data loader and then multiply it with the pixelwise optical flow loss. The key challenge would probably be ensuring that the segmentation is high enough quality.