Setting a proper camera transform

[fbottarel]

Hello everyone, and first of all thank you @mikedh for the insane work put into this library. I am trying to use this in a few projects, especially for raycasting and some quick renders.

I have been trying for a while to set a custom camera transform, but I only managed to get myself a headache since there seems to be some weird convention going on with transform matrices. My questions are:

1. How can I use the trimesh.scene.set_camera method to define a camera viewpoint starting from a regular 4x4 matrix containing camera pose and 3D position with respect to the world reference frame?
2. If there is a differerent convention in defining such reference frames, where can I find more info? Is there anything in the trimesh documentation?

Thank you in advance F

[marcofariasmx]

Also interested in such solution please. Thank you @mikedh and to all the contributors!

[Hippogriff]

@fbottarel @marcofariasmx Have you found the solution?

[fbottarel]

Hey @Hippogriff, take a look at #447, it was useful in shedding some light on my issues. In my case, I simply didn't know the conventions used in camera transforms when dealing with 3D graphics.

[Hippogriff]

@fbottarel I found the solution. The camera_transform is world to camera transformation matrix (or reverse, I can't tell to be honest, try inverse if it doesn't work). But the weird thing is that you need to rotate your mesh with the following transformation before feeding into the rendering pipeline:

    extra = np.eye(4)
    extra[0, 0] = 0
    extra[0, 1] = 1
    extra[1, 0] = -1
    extra[1, 1] = 0

So if you got pyrender rendering working, then simply pre multily the camera_pose matrix and that's your camera_transform that trimesh expects.

[barvin04]

Hello everyone, and first of all thank you @mikedh for the insane work put into this library. I am trying to use this in a few projects, especially for raycasting and some quick renders.

I have been trying for a while to set a custom camera transform, but I only managed to get myself a headache since there seems to be some weird convention going on with transform matrices. My questions are:

1. How can I use the trimesh.scene.set_camera method to define a camera viewpoint starting from a regular 4x4 matrix containing camera pose and 3D position with respect to the world reference frame?
2. If there is a differerent convention in defining such reference frames, where can I find more info? Is there anything in the trimesh documentation?

Thank you in advance F

Yes, there are some conventions to be followed. Different platforms use different coordinate system. (example)

For LineMOD dataset, I found that you need to multiply y and z axis transformations with -1 to correctly render in Trimesh.

r  = np.multiply( r, np.array([[1, 1, 1], [-1, -1, -1], [-1, -1, -1]])) #rotation
t = np.multiply( t , np.array([1, -1, -1])) #translation