search

zju3dv / 4K4D

[CVPR 2024] 4K4D: Real-Time 4D View Synthesis at 4K Resolution
https://zju3dv.github.io/4k4d/
Other
1.56k stars 67 forks source link

Extraction of depth map and normal map #30

Open gwang-kim opened 5 months ago

gwang-kim commented 5 months ago

Hi, Thank you for the great work! I'm assuming that we can render depth and normal map from the learned point clouds. Could you give any tips for extracting depth and normal map from the learned models?

Thanks, Bradley

dendenxu commented 4 months ago

Hi, the depth map extraction is already supported and enabled by default. For offline rendered depth, you can find the rendered depth image if you've run some training at data/result/xxx/DEPTH. For online rendering, you can find the depth visualization in the Rendering tab and Visualization node.

For normal maps, I think it should be possible to have one by applying normal estimation methods on the point clouds directly (the quality might suffer). But generally, the point cloud representation of 4K4D doesn't incorporate normal extraction directly. Reference implementation: https://github.com/zju3dv/4K4D/blob/a79cba4cf384bb3bb0adbdae7b60a305035f8bed/easyvolcap/utils/gl_utils.py#L294

Another interesting directly is to compute the normal map as the gradient of the depth map. I think it should lead to better quality than the method I mentioned above.