search

google / lasr

Code for "LASR: Learning Articulated Shape Reconstruction from a Monocular Video". CVPR 2021.
https://lasr-google.github.io/
Apache License 2.0
169 stars 17 forks source link

Question on Flow preprocessing #15

Open kakashiht opened 2 years ago

kakashiht commented 2 years ago

Hi,

Thank you for open-sourcing your awesome work.

Could you explain on what is going on with the flow pre-processing below?

https://github.com/google/lasr/blob/492fa417bce7ec8743da80dda267320ade153873/dataloader/vidbase.py#L145-L151

Why is this preferred over a simple MSE penalty over raw flow fields?

Thanks!!

gengshan-y commented 2 years ago

Hi, this is to account for image cropping in a pair of frames.

The following converts flow in the raw image coordinate to the cropped image coordinate. It is essential. https://github.com/google/lasr/blob/492fa417bce7ec8743da80dda267320ade153873/dataloader/vidbase.py#L145-L147

The following converts flow to normalized device, assuming image coordinate is in (-1,1). This is optional as long as you modify render function to render pixel unit flow. https://github.com/google/lasr/blob/492fa417bce7ec8743da80dda267320ade153873/dataloader/vidbase.py#L149-L151