Rotation issue

[amir989]

Great work,

If you rotate the camera in z axis, then the points will be in wrong position. but this is not happening in the Realsense sample scene. I think the problem is happening because the camera image also rotates. but, i couldnt figure out how to fix this issue in your sample! any thoughts?

Cheers!

[keijiro]

Could you attach a screenshot or video that explains what's happening on your side? I'm not sure if the "camera" means the RealSense camera (physical device) or a Unity Camera component.

[amir989]

physical devise. sorry i should have been more clear!

Here is a video. if you look at the blue object, you will see it will completely moves to the left in the world when i physically rotate the sensor. I hope this help!

[keijiro]

Thanks for the video. Which device are you using? I only tested it with a D415 + T265 combo.

[amir989]

Im using D435i + T265.

I just figured out that Intel's sample also has same problem. I move my mouse to the corner of the blue mattress the second time im rotating the camera. and you can see the mattress moving in space when i rotate the camera. I still think the problem is in camera image rotation. if you look at the right corner, you will see that the image is rotating when im rotating the camera. like the blue mattress turning to left. this should NOT happen. if you look at any other platforms, like Tango, ARCore or ARKit. no matter how you rotate the camera, the camera image should'nt rotate. so the mattress should still look up!

Let me know what you think!

[keijiro]

this should NOT happen

This should happen because D4xx is just a depth camera. If you rotate the camera, the image should ratote.

My Capture.unity transforms camera-space point cloud into the world space using the positional data given from T265. I adjusted it with D415 to reduce the error as much as possible, but it still drifts when rotating the camera. I guess that it become worse when using other cameras like D435i.

[amir989]

I'm not an expert in this. could you please explain what do you mean by adjusting the tracking with the D camera? is it more than translating the points to the Unity world?

I was looking at Tango API for transferring points to Unity world. I couldn't understand what is the m_unityWorldTStartService * startServiceTDevice part. I thought sharing this with you. Maybe it can help!!

Also, thanks again for your time and your amazing demos.

[keijiro]

I'm not an expert in this. could you please explain what do you mean by adjusting the tracking with the D camera? is it more than translating the points to the Unity world?

It's just translating the points, but it's important how you do it. I determined the variables based on Intel's datasheet and a 3D-printed custom bracket (see below).

https://www.tinkercad.com/things/6ZrIcJwBqyC-realsense-t265-and-d415-mount-ver-2

I think you have to re-adjust these variables if you're using different devices.

I was looking at Tango API for transferring points to Unity world. I couldn't understand what is the m_unityWorldTStartService * startServiceTDevice part. I thought sharing this with you. Maybe it can help!!

Sorry, that's not relevant to the topic.

[amir989]

Alright, ill see if i can find a way to fix this. it would be helpful if you can share a video from your demo with your sensor while you are rotating your camera in z axis. I'm just interested to see how much offset your sensor has!