Problem. Where is Projector intrinsic and extrinsic data ?

[moonsh]

Hello.

I am trying to projection mapping 3D points in the depth camera to projector . However, I have faced problem.

I have found same issue here and seen answer.

https://github.com/Kinect/RoomAliveToolkit/issues/13 [[[ I assume you mean a 3D point in the depth camera coordinate frame? This is a simple coordinate transform, i.e. 3D point x' in projector coords is A * x, where A is the projector's 4x4 pose matrix and x is the (homogenous) point in the depth camera. x' can then be projected to projector image coordinates using the intrinsics (look for the 'Project' function in the code). ]]]

In my computer, after calibration, it only shows camera matrix and camera pose like a bottom figure. I thought these are calibration results of projector. However, using these data, 3d points are not projected to object.

I use one main display and the projector attached to the Pc.

Many thanks

[thundercarrot]

The camera matrix listed there is the camera matrix for the projector. The camera pose is the pose of the depth camera (during the optimization, projector is the center of the coordinate system, and we solve for the depth camera poses). I believe if you invert the pose of the depth camera you will have the pose of the projector.

In any case, it is probably better to be reading these values from the .xml file rather than the debug output.

The projection mapping step is all in the shader code; it can be a little difficult to follow and I'm thinking it would be nice to have an example broken out in code just for illustrative purposes. Kind of like the "CoordinateMappingExample" in KinectServer.

[moonsh]

Thanks a lot. I succeed projection mapping, however the result is not much great. Errors are found at least 3cm between object and projection mapping image. These errors are from Kinect itself accuracy ??

[thundercarrot]

Can you zip up all the files in the calibration directory and put them in a place where I can download them? I will take a look.

[moonsh]

Here it is. I am not sure it will be helpful .

https://github.com/moonsh/download

[rolandsmeenk]

Looks like the calibration of the projector does not exactly match the real projector. The purple frustum represents the projector and the magenta frustum is the Kinect depth view. The purple lines should intersect the corners of the projected image if the calibration is perfect.

Looks like calibration fails with the data you provided. The projector is calibrated in the ensemble, but apparently that is the result from a different calibration pass. Try calibration with some more white objects in the view.

[thundercarrot]

It looks like the calibration was performed with the statue head. In these images the Gray code decoding appears to fail because the projected image is quite small in the Kinect color camera image. Try moving the Kinect closer, or moving the projector farther away. For best results I would set things up so that the Kinect color image can see all of the projected image, and that the projected image about fills the Kinect color image.

[ojack]

@thundercarrot it would be really helpful to have a coordinate mapping example :)

I tried to follow the shader code, but I think I am missing a piece. I would like to use the callibration to map a point in the depth image to a 2D projector coordinate. Do these steps make sense?: --load callibration into program and create projectorWorldViewProjection matrix (following code in ProjectionMappingSample.cs) --for a given point (x, y) in the depth image, use depthCameraTable to get depthCameraPoint (following code in DepthAndProjectiveTextureVS shader) --multiply projectorWorldViewProjectionMatrix by depthCameraPoint ... then I am not sure how to use the resulting point/how to get the 2d projector point....:/ thanks!

[thundercarrot]

Assuming you got all the details correct, you will now have a point that lies in the view volume, where x, y in [-1, 1], z in [0, 1], x right, y up, z forward. To turn that into projector image coordinates, drop z, and do a simple change of coordinates on x & y from [-1,1] to your projector image dimensions. You may need to flip y, since as noted above the view volume has x right and y up.