Camera Intrinsics for Matterport Dataset

[ruddradev]

Hello,

When trying to run the Active Neural SLAM with Direct Depth to Ego Map modification on Matterport Dataset. I found that the ego views transposed with the Robot Pose are not registering correctly with the Occupancy Grid, while that is not a problem with the Gibson Dataset. It appears as if there is some distortion in the camera intrinsics of matterport Dataset since in the Ego Map, there is some distortion in walls (appearing curved). I could not find any details in the Matterport Dataset or on the website. Could you please help?

I tried correcting the FOV, frame height and width, as well as other parameters related to the intrinsics.

Frame width: 640 Frame Height: 480 FOV: 79 Camera Angle: 0 degrees Camera Height: 0.88 m

xc = (width - 1.) / 2. zc = (height - 1.) / 2. f = (width / 2.) / np.tan(np.deg2rad(fov / 2.))

[devendrachaplot]

Hi, It is difficult for me to suggest what might be going wrong. In our experience, the code works without any change for matterport3D. Could you list the steps you took to test with Matterport environments?

[ruddradev]

Hi Devendra,

Thanks for getting back to me.

I had heavily modified the Active Neural SLAM code in the above example so I tried the original code now.

I tried cloning the repository as it is and running it on mp3d. There were minor tweaks I had to do, post which I get the error - Trade::AbstractImporter::image2D(): index out of range

The modifications I made, the inference command I ran and the full error I got are attached. There are no errors when using the Gibson dataset for Habitat.

I appreciate your help in solving this issue.

Warm Regards, Ruddra

Steps required to run on mp3d.txt error on mp3d.txt run_on_mp3d.txt

[devendrachaplot]

Hi,

Just confirming if I understand you correctly, you had to clone a fresh version of habitat-api inside the env folder to resolve the error while running with matterport dataset? Did you have to do any other change?

[ruddradev]

Hello,

Yes you're right. I had to do 4 steps to run the code on Mp3d but still getting the above error. One of them was cloning the habitat_api. The steps I followed are there in 'Steps required to run on Mp3d.txt' I have attached above.

Post which I am getting the error on Matterport but not on Gibson.

[ruddradev]

Hello,

I was able to take help from someone else to replicate this issue. It is not an issue for all installations. The second attempt worked without issues. I’ll reconfirm and add comments here.

Thanks for your support.

[ruddradev]

Closing for now. Will reopen if I find further info.

[neilzhuu]

Hi, I got the same error when running with mp3d dataset. Did you solve it?

[ruddradev]

Hi @Junior-Zhu : For me it simply worked when we installed the dependencies on another machine. So could be something with the dependencies.