mbanani / probe3d

[CVPR 2024] Probing the 3D Awareness of Visual Foundation Models
MIT License
237 stars 11 forks source link

Difference between the grid points and the projected into 2D points #7

Open carlosjedwab opened 4 months ago

carlosjedwab commented 4 months ago

I am trying to recreate the point correspondence between two images from the navi dataset (not just the error evaluation, but the actual image mapping), but there seems to be a translation between where the projected points should be and where the grid points actually are. This error seems to be different for each image. Where exactly does this translation come from and how do I correct it for both image pairs? I suspect there might be some camera attribute that was not considered in the computations within the existing code, but I wasn't able to find anything.

Here is a demonstration of the error between the grid points uv_0 and the 3D to 2D points xy_0:

image

Here is the image with the xy_0 points for reference:

image

And here is the image with the uv_0 points:

image

mbanani commented 1 month ago

Sorry for the delayed response. I didn't notice the issue when it showed up. Based on this images you attached, it seems that the difference is simple due to taking an image plane (uv) which goes from (0,0), (width, height), or if we center the image plane at 0. I would need to look into it more though. Where you able to fix the problem or is it still an issue?