Question to Azure Kinect transformation

[tteichmeister]

Hi,

I have a question:

I get a 3D joint point from the azure kinect. Afterwards I want to draw a line in every dimension. (a straight line x, y, z) I already figured out that the coordinate system is different which mean that I have to do the following:

var xVector = MathNet.Spatial.Euclidean.UnitVector3D.Create(0, -1, 0); // to the right
var yVector = MathNet.Spatial.Euclidean.UnitVector3D.Create(0, 0, 1); // to the top
var zVector = MathNet.Spatial.Euclidean.UnitVector3D.Create(1, 0, 0); // to the front

Afterwards I construct the lines and transform the line back to the color image.

var endPointX = point + xVector.ScaleBy(1);
var endPointY = point + yVector.ScaleBy(1);
var endPointZ = point + zVector.ScaleBy(1);

var xPoint = depthDeviceCalibrationInfo.ToColorSpace(endPointX);
var yPoint = depthDeviceCalibrationInfo.ToColorSpace(endPointY);
var zPoint = depthDeviceCalibrationInfo.ToColorSpace(endPointZ);

X and Y looks good to me. But the zVector is wrong. I'm expecting that the line should not be visible since the line is a point in this direction.

But Z doesn't look like that. It seems to me that the line goes down. I would expect the following:

Do I miss something, or do I have to apply a transformation in this case ?

Sorry but I cannot figure out where the problem might be ?

Thanks in advance.

[sandrist]

The default coordinate system basis assumed by MathNet (and thus by Psi) is that the X-axis points "forward", the Y-axis points "left", and the Z-axis points "Up". So for the Azure Kinect Psi component, you can imagine the X-axis pointing from the center of the device forward through the lens, the Y-axis pointing to the device's left, and the Z-axis pointing up.

When we call that ToColorSpace method, we're taking a 3d point in the coordinate system of the device, transforming into a 3d point in the coordinate system of the color camera, and then projecting through the color camera intrinsics to get a 2d image point.

So for example, if I take the 3D point of my nose (obtained by the body tracker), and project to color camera space (along with a shift in the direction of UnitVector3D.XAxis, UnitVector3D.YAxis, and UnitVector3D.ZAxis), I see something like the following: The X-axis is pointing from my nose (in black) into my head (in red) in the same direction that the camera is pointed. My nose isn't perfectly centered in front of the camera, so you see a bit of an offset in the axis pointing into my head due to the perspective transformation.

If I change my axis directions into what you described, I see the following: X points to the right, Y points up, and Z points into my head (again viewed at a bit of an angle and projected through the perspective transformation).

I'm not sure exactly what transformation you're hoping to accomplish, but hopefully that explanation helps inform a mental model of what's going on and why you're seeing the results you are seeing.

[tteichmeister]

@sandrist Sorry for the very late response. Your explanation helped a lot, it was my fault. You're right I was confused by the visualization, but obviously is right in this case.

Thank you very much for your detailed explanation and for answering my question.

Best regards

[sandrist]

Glad it helped! I will close this issue for now, but feel free to reopen if you have any follow-up questions.