search

pmj110119 / RenderOcc

[ICRA 2024] RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervision. (Former version: UniOcc)
432 stars 24 forks source link

Question about Ray Transformation #24

Closed rayeeli closed 11 months ago

rayeeli commented 11 months ago

Hi author,

Thank you for your open source work!

I noticed that there is a bda transformation in https://github.com/pmj110119/RenderOcc/blob/main/mmdet3d/models/nerf/nerf_head.py#L79

I'm confused about this. What is this used for? Because the rendered depth map and semantic map are compared directly to the original depth map and semantic map, not to the data-augemented one. So keeping the sampled ray_pts is enough. Please correct me if I have misunderstood.

pmj110119 commented 11 months ago

Hi @rayeeli , you are meticulous.

BDA-aug will change the voxel coordinate system by performing random horizontal or vertical flips at the BEV level. During rendering, we need to compute the xyz coordinates of ray_pts for voxel sampling. Therefore, the same transformations must be applied to align with the modified voxel coordinate system.

Hope this is helpful to you.

rayeeli commented 11 months ago

thanks a lot