search

dcharatan / flowmap

[3DV 2025] Code for "FlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient Descent" by Cameron Smith*, David Charatan*, Ayush Tewari, and Vincent Sitzmann
https://cameronosmith.github.io/flowmap/
MIT License
893 stars 87 forks source link

Custom data #24

Closed tonyyu0822 closed 6 months ago

tonyyu0822 commented 6 months ago

hi, im trying to run flowmap on custom data. I wonder what config i need to change to do that? I am getting RuntimeError: stack expects each tensor to be equal size, but got [3, 1008, 756] at entry 0 and [3, 864, 1152] at entry 1

when I simply use dataset: images

Thanks

dcharatan commented 6 months ago

FlowMap is designed to run on video sequences, so it assumes that all frames have the same dimensions. It can't directly be used with differing image dimensions for three reasons:

  1. As far as I know, RAFT (the optical flow estimation network FlowMap uses) only works when the inputs have the same size.
  2. FlowMap assumes that the focal length is shared across all frames.
  3. When images have differing dimensions, it usually means they come from different cameras. FlowMap assumes that the data is sequential (i.e., video-like), and it can be difficult to find a sequential ordering that makes sense for arbitrary image collections.

In theory, FlowMap could be adapted to use "video-like" data from several different cameras, but you would probably have to use point tracking/classical feature matching to establish correspondence and learn one focal length per camera or per image.