search

nianticlabs / simplerecon

[ECCV 2022] SimpleRecon: 3D Reconstruction Without 3D Convolutions
Other
1.3k stars 121 forks source link

Reconstruction using camera pose from ORB-SLAM2 #40

Open BenedictusKent opened 1 year ago

BenedictusKent commented 1 year ago

Hi, I'm wondering if anyone tried to do reconstruction using camera pose from ORB-SLAM2?

I modified the ORB-SLAM2 code to capture each frame image, each frame's camera pose, and I set intrinsic the same for all frame because I need to reconstruct using only monocular camera (and ORB-SLAM2 use a fixed camera intrinsic as far as I know).

I formatted the result as similar as the ARKit dataset format and feed the result to the model. But for some reason, the resulted reconstruction is terrible as shown in the image below. snapshot-bad

I tried to scan the same room with the ios-logger, and it worked fine as you can see from the image below. snapshot-ok

I thought the problem is because the coordinate system of ORB-SLAM2 and ARKit is different. ARKit: +x to right, +y to up, +z forward ORB-SLAM2: +x to right, +y to down, +z to forward

I tried to convert ORB-SLAM2 coordinate system to ARKit by multiplying a rotation matrix (180 degrees) with the camera pose. On paper, the coordinate system should be aligned already. But the result is terrible, basically same as above.

Does anyone have any suggestions or clues what I should do?

Gatsby23 commented 1 year ago

I think there maybe two reasons(just guess, not right):

  1. The pose from ORB-SLAM2 should be relative pose without the absolute scale which may drove the system worse.
  2. The pose from airkit is more accurate than the one you estimated(from ORB-SLAM2)