How to filter the epipolar features

[indigopyj]

I appreciate your work and I am very interested in it! By the way, I saw your code and wonder where the code adding an inductive bias for epipolar constraints is. I found "gnt/data_loaders/data_verifier.py" but this doesn't apply to actual model implementation. Or is it just omitted because all the datasets you use already satisfy the epipolar constraints?

[MukundVarmaT]

Hi, Thank you for your interest in our work. Given camera poses, we can always estimate epipolar projections - that may or may not be valid. https://github.com/VITA-Group/GNT/blob/main/gnt/projection.py#L52

We argue that our view transformer learns to account for such inconsistencies and aggregate information from required projections. For example, below we replicate Fig. 6 from the paper, but corrupt the source views with images from the opposite direction. We observe that our view transformer does not pay attention to these source views at all.