facebookresearch / pytorch3d

PyTorch3D is FAIR's library of reusable components for deep learning with 3D data
https://pytorch3d.org/
Other
8.7k stars 1.3k forks source link

How to render an image with the correct camera position ? #1809

Open MatteoMarengo opened 3 months ago

MatteoMarengo commented 3 months ago

Hello,

I would need help on a specific task.

I am working on the nuScenes dataset and I want to render a mesh with the right camera viewpoint.

I have a texture map (a .png file) and I define my 2D mesh as a rectangle consisting of two triangles.

Here is my .obj file :

mtllib simple_mesh.mtl

v 33.11250686645508 -50.19285202026367 0.0 v 133.11233520507812 -50.19285202026367 0.0 v 133.11233520507812 49.67078399658203 0.0 v 33.11250686645508 49.67078399658203 0.0 vt 0.0 0.0 vt 1.0 0.0 vt 1.0 1.0 vt 0.0 1.0 f 1/1 2/2 3/3 f 1/1 3/3 4/4

I also have the camera parameters:

camera_params = { "frames": [ { "file_path": "samples/CAM_FRONT/n008-2018-08-27-11-48-51-0400__CAM_FRONT__1535385092112404.jpg", "time": 1535385092112404, "transform_matrix": [ [0.009996810324503643, -0.9998783580101878, -0.01197217466409921, 0.00912502329035999], [0.008842632746802154, 0.012060700994871545, -0.9998881674155458, 1.4819740829478583], [0.9999109318481532, 0.00988982681163363, 0.008962125651585906, -1.4544289915843547], [0.0, 0.0, 0.0, 1.0] ], "intrinsics": [ [1252.8131021185304, 0.0, 826.588114781398], [0.0, 1252.8131021185304, 469.9846626224581], [0.0, 0.0, 1.0] ], "is_key_frame": True } ] }

Therefore I can have my R and my T, but when I am doing the rendering using the MeshRenderer (I have defined cameras with the correct R and T), I only observe a white background.

Thank you so much !