search

oscarmcnulty / gta-3d-dataset

A dataset of 2D imagery, 3D point cloud data, and 3D vehicle bounding box labels all generated using the Grand Theft Auto 5 game engine.
134 stars 15 forks source link

Depth scale #5

Open VladimirYugay opened 3 years ago

VladimirYugay commented 3 years ago

Hey there,

Thanks for your work.

I'm currently reading depth maps with this and obtain them in .raw format. Do you know any way to convert it properly to the depth in meters? The values inside are integer values, and just normalizing it to [0, 1] range doesn't make much sense.

oscarmcnulty commented 3 years ago

The depth matrix you are seeing is likely in the "clip" space. To transform to the view space (regular cartesian coords relative to camera position) you need to transform it using the "projection" matrix which should also be captured from the game engine. The units for the view space coordinate system and whether you need to scale the raw depth values to [0, 1] before transforming would depend on how the projection matrix and depth matrix are captured/stored.

To transform clip space to view space for this dataset you can see the example code here

There is some detail in the thesis on page 26 about what clip space is and how the rendering pipeline works.