Problem about alignment

[xufengfan96]

Hello author, I using Intrinsic3D on my dataset.And here is the color images and depth maps. And I also test whether color images and depth maps are aligned.And here is the result.I think they are aligned. However when I try to fusion them.There is some thing wrong with the color.So do you have any advice?

[robmaier]

I would guess that something's wrong with the extrinsic camera poses (or the color camera intrinsics), but not sure. You could look at the fusion results with just 1 or 2 frame, i.e. setting max_frames: "1" in sensor.yml, and see whether things are already wrong there.

[xufengfan96]

I would guess that something's wrong with the extrinsic camera poses (or the color camera intrinsics), but not sure. You could look at the fusion results with just 1 or 2 frame, i.e. setting max_frames: "1" in sensor.yml, and see whether things are already wrong there.

I use only 1 frame,and here is the result: And I don't think the colorIntrinsics and depthIntrinsics is wrong. So is that means my pose was not accurate.

[robmaier]

A translation tx of 224.145615 looks at least suspicious at a first glance from its magnitude (but might also be correct nevertheless).

[xufengfan96]

A translation tx of 224.145615 looks at least suspicious at a first glance from its magnitude (but might also be correct nevertheless).

The last two images are the parameters about my tof camera.It's not the pose.I only use intrinsic matrix.Do I need to upload my pose file.

[robmaier]

I see. Another option might be that you need to use inverse the pose instead?

[robmaier]

Also, the center pixels of both camera intrinsics (depth and color) seem to be quite off from the actual center pixel in the middle (which should be near (640, 480)). Maybe you can verify that this is correct.

[xufengfan96]

I am sorry I took the time to try out your suggestions.Is inverse the pose means inverse matrix?I use inverse matrix but it doesn't work.About center pixels, I don't really understand what you mean.The size of the images I took is 1280x960.And the camera intrinsics will not be changed,right?

[xufengfan96]

What's the meaning about center pixels?

[robmaier]

You should be able to compute the inverse pose as the matrix inverse, yes.

• Which sensor did you use to collect the RGB-D data?
• Where did you get the intrinsic calibration from? We are assuming the pinhole camera model with center pixel cx,cy and focal length fx,fy here.

[xufengfan96]

I use a TOF camera from a Chinese company.Here is the website about my camera.And it will provide a script to print the internal matrix as you can see in my screenshot before.

[xufengfan96]

https://www.percipio.xyz/dev_detail/?model_id=299

[robmaier]

Wait, are you using just a single RGB-D frame? Then the extrinsics can be the identity matrix anyhow. But color and depth must be pre-registered. (are you using the extrinsics between color and depth sensor here as extrinsics? This doesn't work ...) But it seems to me that are no problems related to the Intrinsic3D code base, but only related to the RGB-D input data and its preprocessing.

[xufengfan96]

Wait, are you using just a single RGB-D frame? Then the extrinsics can be the identity matrix anyhow. But color and depth must be pre-registered. (are you using the extrinsics between color and depth sensor here as extrinsics? This doesn't work ...) But it seems to me that are no problems related to the Intrinsic3D code base, but only related to the RGB-D input data and its preprocessing.

I also use several frame. Is the extrinsics means the pose. I calculate poses by myselfmyself. And I use identity matrix as the first pose. So do you think it's the reason that make bad result.