Closed kevinchiu19 closed 2 months ago
Depth can either be defined as distance along the ray or as Z depth (distance along the camera look vector/Z coordinate in camera space). Since depth is predicted by a neural network, the convention that's being used doesn't matter—the network will simply learn whatever convention is being used.
If you want to switch to the other convention (Z depth), you can replace this line with the following:
directions = directions / directions[..., -1:]
This will normalize by the Z coordinate instead of by ray length.
Depth can either be defined as distance along the ray or as Z depth (distance along the camera look vector/Z coordinate in camera space). Since depth is predicted by a neural network, the convention that's being used doesn't matter—the network will simply learn whatever convention is being used.
If you want to switch to the other convention (Z depth), you can replace this line with the following:
directions = directions / directions[..., -1:]
This will normalize by the Z coordinate instead of by ray length.
OK, thank you for the detailed explanation, and thank you again for your great work!
Thanks for sharing your great effort and work.
I have a little question here in src/model/encoder/common/gaussian_adapter.py:
Compute Gaussian means.
The directions used in the calculation here has been normalized, which will cause the calculation result to not be the correct world coordinate system.
For example, in the autonomous driving scene, the xyz of the point cloud is directly used as the gaussian position. Here, normalization and non-normalization are used. The results are as shown in the figure below.![image](https://github.com/dcharatan/pixelsplat/assets/52447673/250628d5-3e26-4e69-b15a-19c1179631da)