Depth map multiplied by alpha values

[DogeSquad]

Hi there,

I have noticed that the output depth map is somehow multiplied by the alpha values. Am I missing something or is this not intentional? Because fading out the depth via the alpha values pushes pixels closer to the camera.

On the left is the original depth map. On the right is the corrected depth map (division by alpha values).

[slothfulxtx]

Hi, My advice on the depth map is that you should regularize the output alpha map with a ground truth object mask during training, and directly use the output depth map instead of the normalized one. Maybe this solution can help.

[2019213039]

Hello, thanks for open source. What is the rendered alpha image used for?

Here are the original image:

Here are the mask:

Here are the depth label: Here are the depth results I got when training on the waymo dataset:

Here are the alpha result:

From the rendered depth map, the odd thing is that for the close up white Land Rover, the rendered result sets the distance of the vehicle to the farthest distance ([1.0, 1.0, 1.0]). Can you tell me why?

[slothfulxtx]

Sorry for the delayed response. It's quite interesting that your depth map is highly related to the semantic information of the rendered image. My advice is that use your gt depth and gt mask to supervise the 3D GS during optimization. Setting the weight of depth loss and alpha loss as 1.0 may be helpful. @2019213039 Further discussion can be sent to my email (xutx21@mails.tsinghua.edu.cn), in which way I can see your message timely.